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Preface 


In  the  spring  of  2009,  the  Deputy  Assistant  Secretary  for  Acquisition  Integration,  Office  of 
the  Assistant  Secretary  of  the  Air  Force  for  Acquisition,  and  the  Deputy  Assistant  Secretary  for 
Contracting,  Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition,  asked  RAND 
Project  AIR  FORCE  to  assist  the  Air  Force  in  enhancing  its  contract  cost/price  analysis  capa¬ 
bility  in  order  to  improve  the  outcomes  of  its  major  weapon  system  development  contract 
negotiations. 

Midway  through  the  study,  the  sponsors  expanded  its  scope  to  include  the  feasibility 
of  improving  cost  realism  assessments  for  development  contracts  and  developing  a  capability 
for  should-cost  reviews.  Federal  Acquisition  Regulation  (FAR)  15.407  defines  the  should-cost 
review  as  a  specialized  form  of  contract  cost  analysis  that  seeks  to  improve  contractor  efficiency 
and  lower  contract  cost  to  the  government. 

In  September  2010,  after  the  research  for  this  study  was  completed  earlier  in  that  year,  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  (USD[AT&L])  issued 
guidance  to  the  Department  of  Defense  (DoD)  that  mandates  the  use  of  should-cost  analysis 
for  major  programs  (Carter,  2010a).  In  November  2010,  USD(AT&L)  issued  an  implementa¬ 
tion  memorandum  for  the  September  2010  guidance  that  directed  DoD  components  to  estab¬ 
lish  should-cost  targets  for  Acquisition  Category  I,  II,  and  III  programs  as  they  are  considered 
for  acquisition  milestone  decisions  (Carter,  2010b).  USD(AT&L)  provided  further  clarifica¬ 
tion  in  an  April  2011  memorandum  (Carter  and  Hale,  2011)  that  explained  that  should-cost 
estimates  can  be  developed  in  three  ways:  (1)  conducting  a  detailed,  bottom-up  analysis,  (2) 
identifying  discrete  reductions  from  will-cost  estimates,  and  (3)  using  competitive  contracting 
and  contract  negotiations. 

Because  of  the  widespread  interest  throughout  DoD  in  should-cost  reviews  after  the  issu¬ 
ance  of  the  USD(AT&L)  memos,  RAND  extracted  and  expanded  the  research  on  should- 
cost  reviews  from  the  larger  study  into  this  separate  technical  report.  The  RAND  research 
addressed  should-cost  reviews  as  described  in  the  FAR  and  similar  analyses,  although  many 
of  the  historical  examples  did  not  fully  meet  the  FAR  description.  The  historical  experiences 
reviewed  in  this  report  encompass  the  first  two  approaches  found  in  the  April  2011  guidance. 
This  publication  focuses  on  RAND’s  investigation  of  previous  should-cost  reviews,  which 
addressed  two  key  questions: 

•  What  has  been  the  Air  Force  experience  with  should-cost  reviews? 

•  How  can  the  Air  Force  enhance  its  capability  to  conduct  should-cost  reviews? 

This  report  should  interest  those  involved  in  acquiring  defense  goods  and  services  at  the 
most  favorable  prices,  particularly  acquisition  personnel  who  may  be  involved  in  planning 
or  conducting  should-cost  reviews,  including  program  managers,  contracting  personnel,  cost 
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estimators,  industrial  engineers,  and  subject-matter  experts  in  the  technologies  used  in  major 
weapon  systems. 

This  research  was  conducted  within  the  Resource  Management  Program  of  RAND  Proj¬ 
ect  AIR  FORCE  as  part  of  the  fiscal  year  2009-2010  study  “Enhancing  the  Price  Analysis 
Capability  of  the  U.S.  Air  Force  for  Development  Contracts.” 


RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corporation,  is  the  U.S.  Air 
Force’s  federally  funded  research  and  development  center  for  studies  and  analyses.  PAF  pro¬ 
vides  the  Air  Force  with  independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  combat  readiness,  and  support  of  current  and  future  air,  space,  and  cyber  forces. 
Research  is  conducted  in  four  programs:  Force  Modernization  and  Employment;  Manpower, 
Personnel,  and  Training;  Resource  Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  website: 
http://www.rand.org/paf 
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Summary 


The  Air  Force,  along  with  the  other  military  services,  has  been  plagued  with  a  history  of  cost 
overruns  in  major  acquisition  programs,  such  as  the  C-5A  and  C-17A  cargo  aircraft  and  F-22A 
and  F-35  fighter  aircraft  programs.  A  series  of  studies  has  shown  that  part  of  that  growth  can 
be  attributed  to  poor  initial  cost  estimates.1  Additionally,  there  has  been  a  long-standing  con¬ 
cern  since  the  1990s  about  the  size,  experience  level,  and  quality  of  the  acquisition  workforce. 
These  concerns,  along  with  the  opportunity  for  the  Air  Force  to  increase  the  size  of  its  acquisi¬ 
tion  workforce  as  part  of  a  larger  Department  of  Defense  (DoD)-wide  initiative,2  prompted 
the  Air  Force  to  ask  RAND  Project  AIR  FORCE  to  help  it  find  ways  to  improve  its  contract 
cost/price  analysis  capability.  The  Air  Force  sponsors  of  this  research  were  particularly  inter¬ 
ested  in  the  feasibility  of  improving  the  Air  Force’s  capability  to  conduct  should-cost  reviews, 
which  are  defined  by  Federal  Acquisition  Regulation  (FAR)  15.407  as  a  special  form  of  con¬ 
tract  cost  analysis.3  This  report  responds  to  the  sponsors’  questions  about  should-cost  reviews. 

The  primary  methodologies  used  to  address  the  questions  were  (1)  interviews  conducted 
with  participants  in  should-cost  reviews  of  Air  Force  programs  and  (2)  reviews  of  the  literature 
on  should-cost  analyses  in  DoD  and  in  commercial  organizations. 


Findings 

The  research  team  posed  the  following  two  questions  regarding  should-cost  reviews: 

•  What  has  been  the  Air  Force  experience  with  should-cost  reviews? 

•  How  can  the  Air  Force  enhance  its  capability  to  conduct  should-cost  reviews? 

What  Has  Been  the  Air  Force  Experience  with  Should-Cost  Reviews? 

The  Air  Force  has  conducted  should-cost  reviews  or  similar  analyses  since  the  1960s,  but  they 
have  been  done  infrequently  and  with  mixed  results.  They  have  been  performed  during  the 
development,  procurement,  and  sustainment  phases  of  major  programs,  ranging  in  scope  from 


1  Various  individuals  and  organizations  have  studied  the  sources  of  cost  growth  in  military  acquisition  programs.  Recent 
RAND  research  on  the  subject  includes  Bolten  et  al.  (2008)  and  Blickstein  et  al.  (2011). 

2  Due  to  budget  pressures  and  reductions  in  end  strength,  the  Air  Force  has  been  unable  to  increase  its  acquisition  work¬ 
force  above  the  level  it  was  at  the  end  of  fiscal  year  2008. 

3  “[Tjhese  reviews  evaluate  the  economy  and  efficiency  of  the  contractor’s  existing  work  force,  methods,  materials,  equip¬ 
ment,  real  property,  operating  systems,  and  management.  .  .  .  The  objective  of  should-cost  reviews  is  to  promote  both  short 
and  long-range  improvements  in  the  contractor’s  economy  and  efficiency  in  order  to  reduce  the  cost  of  performance  of  Gov¬ 
ernment  contracts”  (FAR  15.407-4). 
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reviews  of  the  entire  program  to  smaller  modification  efforts.  The  reviews  require  a  mix  of 
people  with  such  skills  as  cost  estimating,  pricing,  and  contracting  and  subject-matter  exper¬ 
tise  in  manufacturing  or  other  areas  relevant  to  the  particular  program  being  reviewed.  Some 
argue  that  industrial  engineers  are  also  important  members  of  any  review  team. 

The  size  and  composition  of  the  teams  and  the  duration  of  should-cost  reviews  vary  with 
the  scope  of  the  review  and  nature  of  the  program.  For  major  reviews  in  the  past  two  years, 
industrial  engineers  and  price  analysts  and  other  contracting  personnel  have  been  less  repre¬ 
sented  than  in  earlier  Air  Force  experience.  It  is  not  clear  whether  the  skill  mix  required  has 
changed  or  whether  individuals  with  these  skills  were  needed  but  not  available. 

Finding  people  within  DoD  who  have  the  requisite  skills  to  do  should-cost  reviews  on 
a  temporary  basis  poses  challenges  and  diverts  the  individuals  from  their  primary  tasks.  The 
reviews  are  typically  done  infrequently  on  major  acquisition  programs. 

There  is  limited  and  mixed  evidence  as  to  whether  should-cost  reviews  save  money  com¬ 
pared  with  other  forms  of  contract  negotiation.  Ideal  comparisons  to  answer  this  question 
would  be  of  final  contract  prices  on  matched  pairs  of  contracts  (same  contractor,  weapon 
system,  and  contract  type),  where  one  contract  is  negotiated  using  data  from  a  should-cost 
review  and  the  other  contract  is  negotiated  in  some  other  way. 

Potential  savings  identified  during  reviews  may  not  necessarily  be  realized  in  contract 
negotiations  or  achieved  in  the  final  price  to  the  government.  Prominent  practitioners  of 
should-cost  reviews  of  DoD  programs  say  they  can  identify  inefficiencies  and  potential  savings 
during  their  analyses,  but  this  does  not  necessarily  lead  to  savings  to  the  taxpayer.  The  pro¬ 
curing  organization  must  have  the  ability  and  willingness  to  use  the  results  of  the  analysis  in 
contract  negotiations  to  achieve  a  lower  contract  price.  And  the  procuring  organization  must 
further  structure  and  administer  the  contract  in  such  a  way  that  cost  growth  and  contract 
modifications  do  not  negate  the  originally  negotiated  contract  price. 

As  of  the  time  of  the  writing  of  this  report  in  fall  2011,  the  Air  Force  has  no  dedicated 
capability  to  conduct  should-cost  reviews.  The  Air  Force  instead  finds  the  required  individual 
skills  from  across  the  Air  Force  and  among  Air  Force  retirees,  federally  funded  research  and 
development  centers,  support  contractors,  and  other  government  agencies. 

The  ad  hoc  and  infrequent  nature  of  the  reviews,  combined  with  the  absence  of  a  dedi¬ 
cated  capability  to  conduct  them  in  the  Air  Force,  has  resulted  in  an  acquisition  workforce  in 
which  few  people  have  experience  with  the  reviews,  and  there  is  no  core  group  that  can  learn 
from  experience,  retain  lessons  learned,  and  lead  or  provide  training  to  inexperienced  staff. 

How  Can  the  Air  Force  Enhance  Its  Capability  to  Conduct  Should-Cost  Reviews? 

The  Air  Force  needs  to  be  able  to  match  the  demand  for  should-cost  reviews  with  a  supply  of 
people  with  the  appropriate  skills  and  experience  to  conduct  them.  There  are  limited  numbers 
of  personnel  in  some  fields  that  have  traditionally  been  involved  in  should-cost  reviews,  includ¬ 
ing  cost  estimating,  pricing,  and  industrial  engineering.  In  addition,  few  people  in  the  Air 
Force  have  experience  with  conducting  these  reviews. 

A  skeletal  or  virtual  should-cost  review  office,  with  the  capability  to  provide  ad  hoc  train¬ 
ing  to  team  members,  collect  and  retain  lessons  learned,  and  quickly  obtain  personnel  with 
critical  skills,  would  enhance  the  Air  Force’s  capability  to  conduct  these  reviews.  There  are  also 
other,  more  robust  (but  more  costly)  options  for  organizational  support  of  the  capability,  such 
as  an  office  staffed  with  a  core  cadre  of  experts  to  lead  reviews  and  train  other  members  or  an 
office  with  a  large,  dedicated  staff  capable  of  conducting  a  review  of  a  major  program. 
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Recommendations 

We  suggest  four  recommendations  that  should  improve  the  Air  Force’s  ability  to  conduct 
should-cost  reviews. 

Determine  Whether  Should-Cost  Reviews  Result  in  Savings  Compared  with  Other  Forms  of 
Contract  Negotiation 

When  inadequate  competition  exists  to  determine  contract  prices,  the  government  determines 
a  fair  and  reasonable  price  objective  and  attempts  to  reach  agreement  on  the  price  with  the  con¬ 
tractor  through  negotiation.  A  fair  and  reasonable  price  can  be  determined  in  various  ways.  For 
relatively  small  and  routine  procurements,  the  pricing  is  usually  done  by  the  procuring  con¬ 
tracting  officer.  For  larger  contracts  over  certain  dollar  thresholds,4  the  pricing  is  usually  done 
by  pricers,  usually  with  technical  input  from  subject-matter  experts.  In  certain  cases,  generally 
for  large  procurement  programs,  the  pricing  can  be  enhanced  through  a  should-cost  review 
that  informs  contract  negotiations. 

A  detailed  analysis  of  should-cost  review  outcomes  was  beyond  the  scope  of  this  study. 
Information  on  contract  negotiations  and  outcomes  is  retained  by  local  contracting  offices, 
generally  in  paper  records,  and  is  difficult  for  those  outside  the  office  to  obtain.  The  RAND 
research  team  found  little  evidence  in  the  published  research  to  indicate  that  should-cost 
reviews  have  achieved  better  results  than  other  methods  of  contract  pricing  and  negotiation. 
An  important  first  step  in  formulating  a  strategy  for  the  use  of  should-cost  reviews  would  be 
to  determine  whether  they  result  in  savings  compared  with  other  forms  of  contract  negotia¬ 
tion,  and  under  what  circumstances.  This  determination  would  inform  Air  Force  efforts  to 
devote  additional  resources  to  a  should-cost  review  capability.  If  historical  reviews  are  found  to 
produce  results  no  better  than  other  forms  of  contract  pricing  and  negotiating,  the  Air  Force 
should  allocate  resources  accordingly.  If  reviews  using  a  particular  methodology  are  found  to 
be  especially  successful,  resources  should  be  devoted  to  ensuring  that  people  with  the  skills  and 
training  needed  for  the  successful  methodology  are  available. 

Assess  Lessons  Learned  from  Recent  Should-Cost  Reviews 

The  lack  of  a  dedicated  capability  within  the  Air  Force  to  conduct  should-cost  reviews  and  the 
shortage  of  people  within  the  Air  Force  with  experience  in  conducting  such  reviews  suggest  a 
lack  of  broad  and  current  institutional  knowledge  of  the  subject.  The  Air  Force  should  conduct 
an  assessment  of  lessons  learned  from  should-cost  reviews  across  DoD.  The  assessment  should 
address  the  outcomes  of  the  reviews,  whether  people  with  the  required  skills  were  available 
and  what  additional  skills  would  be  needed  for  similar  reviews  in  the  future,  whether  suffi¬ 
cient  time  was  available  for  the  reviews,  and  the  role  of  training  in  the  capability  to  conduct 
the  reviews.  The  Air  Force  may  also  benefit  from  comparing  its  experience  with  should-cost 
against  the  practices  used  by  the  best  commercial  organizations. 


4  A  threshold  of  $10  million  at  three  of  the  Air  Force  product  centers  is  specified  in  the  Air  Force  Materiel  Command 
Federal  Acquisition  Regulation  Supplement. 
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Develop  Training  on  Should-Cost  Reviews 

If  should-cost  reviews  are  found  to  be  effective  in  lowering  contract  prices  relative  to  other 
forms  of  negotiations,  the  Air  Force  should  ensure  that  appropriate  training  is  available  for 
teams.  Training  should  encompass  several  areas: 

•  how  to  assemble  a  team 

•  how  to  conduct  a  should-cost  review 

•  how  to  work  as  a  multidisciplinary  team 

•  lessons  learned  from  previous  reviews,  including  the  possible  finding  that  new  should- 
cost  processes  are  needed  to  address  the  increased  use  of  outsourcing. 

Establish  Databases  of  Cost,  Schedule,  Earned  Value,  and  Technical  Information  Useful  for 
Cost-Estimating  and  Pricing  Activities 

An  industry  best  practice  is  to  develop  a  database  that  can  be  used  strategically  for  contract 
negotiations.  In  the  Air  Force,  relevant  data,  such  as  those  on  cost,  schedule,  earned  value,  and 
technical  issues,  are  collected  by  members  of  various  disciplines  but  not  necessarily  shared. 
These  data  could  be  collected  into  databases  and  used  to  develop  a  better  understanding  of 
industry  standards,  improved  cost-estimating  relationships,  and  documentation  of  contractor 
performance. 
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EMD 

engineering  and  manufacturing  development 
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fixed-price  incentive 
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U.S.  General  Accounting  Office  (now  U.S.  Government  Accountability 
Office) 
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Joint  Strike  Fighter 
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Office  of  the  Secretary  of  Defense 

PAF 

RAND  Project  AIR  FORCE 
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Procedures,  Guidance,  and  Information 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Fogistics 
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Glossary 


administrative  contracting 
officer 

contracting  officer 

A  contracting  officer  (see  definition  below)  who  administers 
contracts. 

A  person  with  the  authority  to  enter  into,  administer,  and / 
or  terminate  contracts  and  make  related  determinations  and 
findings. 

contracting  personnel 

Civilians  in  occupational  series  GS  1102  and  military  per¬ 
sonnel  with  similar  duties,  including  those  who  specialize  in 
negotiating,  administering,  terminating,  and  analyzing  the 
cost  and  price  of  contracts. 

cost  estimators 

Analysts  within  the  business-cost  estimating  career  field, 
who  use  some  techniques  similar  to  those  used  by  pricers  in 
the  contracting  career  field. 

cost-reimbursement  contract 

Contract  for  which  the  government  reimburses  the  contrac¬ 
tor  for  all  reasonable  and  allowable  charges. 

fixed-price  contract 

Contract  for  which  the  contractor  assumes  responsibility  for 
performing  work  at  an  agreed-upon  specified  price. 

pricers 

Formally  known  as  contract  cost/price  analysts,  a  subset  of 
contracting  personnel  who  specialize  in  analyzing  the  cost 
and  price  of  contractor  proposals  to  ensure  that  contracts  are 
awarded  at  fair  and  reasonable  prices. 

will-cost  estimate 

Cost  estimates  informed  by  historical  experience  with  similar 
systems. 
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CHAPTER  ONE 


Introduction 


Background 

This  research  on  pricing  capability  was  motivated  by  the  long-standing  problems  of  cost  growth 
of  major  weapon  system  acquisition  programs  and  the  high  costs  in  general  of  Department  of 
Defense  (DoD)  equipment.1  The  problem  of  cost  growth  has  plagued  DoD  for  several  decades. 
Many  studies  have  documented  the  amount  of  the  growth,  and  some  have  looked  at  the  causes 
of  the  growth.  Many  of  these  studies  have  used  data  from  Selected  Acquisition  Reports  pre¬ 
pared  by  DoD  and  sent  to  Congress  on  major  defense  acquisition  programs.  The  most  recent 
RAND  Corporation  study  of  this  type  that  measured  the  amount  of  the  growth  and  analyzed 
the  sources  of  growth  examined  35  mature  programs  (Bolten  et  ah,  2008).  The  study  found  an 
average  cost  growth  of  57  percent  in  development  and  34  percent  in  production  (after  adjust¬ 
ing  for  changes  in  quantity)  (pp.  28-29).  More  recent  RAND  research  has  explored  in  more 
depth  the  root  causes  of  cost  growth  in  acquisition  programs  (Blickstein  et  ah,  2011). 

Concern  over  cost  growth  and  the  affordability  of  DoD  programs  prompted  Office  of 
the  Secretary  of  Defense  (OSD)  leadership  to  issue  management  initiatives  to  increase  effi¬ 
ciency  and  affordability  in  DoD  acquisition  in  2010.  In  September  2010,  the  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics  (USD[AT&L])  issued  guidance  on  23 
management  initiatives  to  increase  efficiency.  One  of  the  initiatives  requires  program  manag¬ 
ers  of  major  acquisition  programs  to  conduct  should-cost  reviews  of  their  programs  (Carter, 
2010a).  In  November  2010,  USD(AT&L)  issued  an  implementation  memorandum  of  the  Sep¬ 
tember  2010  guidance  (Carter,  2010b).  The  latter  memorandum  directed  DoD  components 
to  establish  should-cost  targets  for  Acquisition  Category  (ACAT)  I,  II,  and  III  programs  as 
they  are  considered  for  acquisition  milestone  decisions.2  In  April  2011,  USD(AT&L)  issued 
additional  guidance  on  the  implementation  of  will-cost  and  should-cost  management  (Carter 
and  Hale,  2011),  which  described  three  methods  that  can  be  used  to  develop  should-cost  esti- 


1  The  term  cost  growth  in  this  context  refers  to  the  increase  in  cost  of  a  given  contract  or  program,  which  is  generally  mea¬ 
sured  from  a  cost  estimate  at  the  start  of  development  to  the  completed  cost  of  the  program.  Many  studies  of  cost  growth, 
including  the  RAND  study  referenced  here,  use  data  from  Selected  Acquisition  Reports,  which  include  the  contractor’s 
cost,  fee,  and  some  associated  government  costs.  The  growth  in  all  these  costs  is  what  is  measured,  and  this  measure  must 
be  understood  as  a  proxy  for  the  growth  experienced  on  any  individual  contract.  Furthermore,  this  type  of  cost  growth 
measure  does  not  take  into  account  modifications  to  contracts  that  reflect  changes  in  the  scope  of  work. 

2  ACAT  I  programs  are  major  acquisition  programs  subject  to  OSD-level  review;  ACAT  II  programs  have  estimated  total 
research,  development,  test,  and  evaluation  (RDT&E)  costs  of  more  than  $140  million  or  total  procurement  costs  of  more 
than  $660  million  in  fiscal  year  (FY)  2000  constant  dollars;  ACAT  III  programs  do  not  meet  the  threshold  for  ACAT  II 
programs  and  can  be  designated  by  the  Component  Acquisition  Executive. 
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mates.3  One  way  is  a  detailed,  bottom-up  analysis.  A  second  way  is  by  identifying  discrete 
reductions  from  will-cost  estimates.  A  third  way  is  through  the  use  of  competitive  contracting 
and  contract  negotiations. 

The  Air  Force  issued  its  own  implementation  guidance  of  the  OSD  initiative  on  will-cost 
and  should-cost  management  in  June  2011  (Morin  and  Van  Buren,  2011).  The  Air  Force  guid¬ 
ance  recognizes  that  the  meaning  of  should-cost  review  addressed  in  the  OSD  and  Air  Force 
guidance  is  broader  than  the  meaning  used  in  the  Federal  Acquisition  Regulation  (FAR)  (dis¬ 
cussed  in  Chapter  Two  of  this  report).  Should-cost  as  part  of  the  management  initiative  encom¬ 
passes  both  government  and  contractor  costs  throughout  a  program’s  life  cycle.  The  Air  Force 
guidance  further  clarifies  that  large,  bottom-up  analyses  of  programs  are  not  required  in  every 
case  and  that  program  managers  are  expected  to  use  their  discretion  in  determining  what  kind 
of  analysis  is  appropriate  for  their  program. 

The  FAR  defines  and  describes  should-cost  reviews  as  specialized  forms  of  cost  analysis 
used  in  support  of  contract  negotiations.  Should-cost  reviews  as  defined  by  the  FAR  have  been 
used  for  decades  in  DoD,  and  therefore  there  is  more  information  available  on  how  they  have 
been  conducted  and  the  results  they  have  achieved.  In  contrast,  the  OSD  guidance  issued 
in  2010  and  2011  and  the  Air  Force  guidance  issued  in  2011  describe  the  use  of  should-cost 
reviews  as  part  of  a  broader  initiative  to  manage  programs  and  costs.  Should-cost  reviews  as 
defined  in  the  FAR  have  a  role  in  the  recent  management  initiative,  but  the  initiative  is  broader 
in  scope  and  intent  than  the  meaning  of  should-cost  in  the  FAR.  The  different  conceptions  of 
should-cost  reviews  and  purposes  of  the  analyses  must  be  kept  in  mind  when  drawing  lessons 
from  historical  experience. 


Purpose 

The  sponsors  of  this  research  asked  RAND  to  examine  the  Air  Force’s  capability  to  con¬ 
duct  should-cost  analyses,  among  other  issues  related  to  contract  pricing.  This  report  addresses 
only  the  subject  of  should-cost  reviews  related  to  contract  pricing.  The  historical  experience 
reviewed  here  does  not  include  Air  Force  experience  with  the  will-cost/should-cost  manage¬ 
ment  initiative  from  OSD  begun  in  2010. 

The  research  team  addressed  two  key  research  questions: 

•  What  has  been  the  Air  Force  experience  with  should-cost  reviews? 

•  How  can  the  Air  Force  enhance  its  capability  to  conduct  should-cost  reviews? 

Recommended  measures  to  improve  outcomes  are  provided  in  the  final  chapter. 


3  Will-cost  estimates  are  cost  estimates  informed  by  historical  experience  with  similar  systems.  In  the  management  initia¬ 
tive,  will-cost  estimates  are  used  to  justify  a  budget  for  the  program  that  provides  enough  resources  to  execute  the  program 
under  normal  conditions. 
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Research  Approach 

The  research  team  used  two  main  sources  of  information  to  address  the  key  issues.  Interviews 
with  acquisition  professionals  involved  in  should-cost  reviews  of  Air  Force  programs  provided 
the  primary  source  of  information  on  recent  experience.  In  addition,  the  team  reviewed  litera¬ 
ture  on  the  use  of  should-cost  reviews  by  DoD  and  commercial  businesses. 

Interviews 

The  RAND  team  conducted  interviews  with  current  and  former  acquisition  professionals, 
primarily  from  the  Air  Force  but  also  from  other  DoD  components  and  from  the  aerospace 
defense  industry.  The  functional  areas  represented  included  pricing,  cost  estimating,  and  engi¬ 
neering.  Interviews  were  semistructured  or  unstructured  because  the  interviewees  had  diverse 
backgrounds  and  experiences  for  which  a  uniform  and  structured  interview  was  not  feasible, 
although  questions  tailored  to  each  interviewee  were  provided  to  them  in  advance.  Prominent 
interview  themes  were 

•  On  what  programs  have  you  participated  in  a  should-cost  review? 

•  What  was  the  composition  of  the  team  in  numbers  of  people  and  their  skills/functional 
areas? 

•  For  reviews  conducted  recently,  what  training  was  provided? 

•  What  was  the  duration  of  the  review? 

•  What  was  the  focus  of  the  review? 

•  What  was  the  outcome  of  the  review? 

Collectively,  the  interviewees  participated  in  should-cost  reviews  of  Air  Force  programs 
from  the  1960s  through  2011,  although  the  interviews  focused  on  reviews  conducted  from  the 
late  1980s.  Interviews  with  people  still  in  the  workforce  were  on  a  not-for-attribution  basis,  so 
the  names  of  such  individuals  are  not  cited  in  this  document. 

The  sample  of  experience  represented  here  is  in  no  way  inclusive  of  all  should-cost  reviews 
conducted  of  Air  Force  programs.  The  interviewees  represent  a  small  sample  of  participants. 
We  are  unaware  of  any  centralized  record  within  the  Air  Force  that  has  information  on  should- 
cost  reviews  that  have  been  conducted,  the  names  of  participants,  or  results,  which  would 
allow  a  more  comprehensive  review  of  experience.  What  little  written  documentation  of  the 
Air  Force  experience  with  should-cost  reviews  that  we  could  find  is  almost  all  open  source.  The 
interviews  and  experiences  reported  here  represent  sources  of  opportunity  rather  than  a  com¬ 
prehensive  report  of  the  subject. 

Literature 

The  team  reviewed  the  literature  on  the  use  of  should-cost  reviews  by  DoD  and  on  the  use  of 
should-cost  and  closely  related  analyses  by  commercial  businesses.  The  literature  on  DoD  use 
of  should-cost  reviews  dates  from  1970  to  1988.  Despite  its  age,  the  literature  highlighted  the 
same  issues  and  problems  raised  by  interviewees  on  the  most  recent  reviews.  The  literature  on 
commercial  use  of  should-cost  and  similar  analyses  included  in  the  report  dates  from  1991  to 
2011. 
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How  the  Report  Is  Organized 

The  next  chapter  contains  a  brief  overview  of  the  contracting  environment  and  process  in  the 
Air  Force,  to  provide  a  context  for  the  reader  about  situations  in  which  should-cost  reviews 
might  be  used.  Chapter  Three  addresses  the  first  of  two  key  questions  examined  during  the 
research:  What  has  been  the  Air  Force  experience  with  should-cost  reviews?  Chapter  Four 
addresses  the  use  of  should-cost  and  similar  analyses  in  commercial  organizations.  Chapter 
Five  addresses  the  second  key  question  examined  in  this  research:  How  can  the  Air  Force 
enhance  its  capability  to  conduct  should-cost  reviews?  Chapter  Six  provides  conclusions  and 
recommendations. 


CHAPTER  TWO 


Background  on  Contract  Pricing  and  Negotiation  in  the 
Department  of  Defense  Acquisition  Environment 


For  major  defense  weapon  systems,  it  is  usually  too  expensive  to  fund  more  than  one  prime 
contractor  to  develop  a  system  in  the  engineering  and  manufacturing  development  (EMD) 
phase.  The  government  almost  always  selects  one  prime  contractor  to  develop  a  weapon  system, 
and  that  same  prime  contractor  normally  becomes  the  sole  source  for  production  of  the  system. 
The  EMD  phase  is  usually  preceded  by  a  “downselect”  at  Milestone  B  from  multiple  competi¬ 
tors  that  are  researching  technologies  and  designs  to  one  prime  contractor  that  will  be  given 
the  opportunity  to  continue  development  into  EMD.1  Competitors  for  these  contracts  have  a 
large  incentive  to  bid  low  in  order  to  win  the  development  contract  because  they  are  normally 
cost-reimbursable  contracts  and  because  DoD  usually  awards  future  production  contracts  only 
to  the  contractor  that  developed  the  weapon  system.  Future  contracts  for  sustainment  of  the 
weapon  system  may  be  at  stake  as  well.  Because  there  is  generally  competition  for  EMD  con¬ 
tracts  and  a  strong  incentive  for  contractors  to  bid  low  to  win  the  contract,  the  government 
does  not  need  to  devote  effort  to  further  reducing  proposed  prices.  Contract  pricing  efforts 
focus  on  ensuring  that  proposed  prices  are  realistic  for  the  work  that  is  to  be  accomplished, 
rather  than  trying  to  reduce  them. 

While  many  or  most  technologies  used  in  the  weapon  system  should  be  mature  by  Mile¬ 
stone  B,  the  major  engineering  and  integration  tasks  required  to  develop,  manufacture,  and 
test  the  final  production  article  are  usually  considerable.  Because  a  lot  of  technological  and  cost 
risks  remain  at  Milestone  B,  it  is  typical  to  have  the  government  bear  those  risks  under  a  cost- 
reimbursement  contract  with  a  multiyear  period  of  performance.2  With  cost-reimbursement 
contracts,  the  government  pays  all  reasonable  and  allowable  costs  incurred  by  the  contractor. 
Even  if  the  government  were  to  devote  effort  to  identifying  inefficiencies  and  negotiating  a 
lower  price  on  a  cost-reimbursement  contract,  it  makes  little  sense  to  do  so.  Unless  an  incen¬ 
tive  fee  arrangement  is  used  on  a  cost-reimbursement  development  contract,  there  is  little  if 
any  motivation  for  the  contractor  to  control  costs  short  of  the  costs  rising  so  much  as  to  risk 
program  cancellation.3 


1  Milestone  B  is  the  decision  point  at  which  significant  development  activities  commence. 

2  Over  the  past  several  decades,  the  government  has  occasionally  tried  to  shift  development  cost  risk  to  contractors 
through  fixed-price  contracts  but  has  had  limited  success. 

3  An  incentive-fee  arrangement  would  reduce  contractor  profit  as  costs  rose  above  a  certain  target  cost,  or  threshold.  But 
the  prospect  of  profits  on  future  production  and  possibly  sustainment  contracts,  after  completion  of  development,  might  be 
more  important  to  the  contractor  than  loss  of  profit  on  the  development  contract.  Competition  to  win  initial  sales  in  order 
to  get  profitable  follow-on  sustainment  and  spares  contracts  is  typical  in  commercial  aviation.  See  Sanders  and  Michaels, 
2011. 
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After  EMD,  the  same  contractor  normally  becomes  the  sole  source  for  production  of 
the  system.  As  sole  supplier,  a  contractor  negotiates  with  its  sole  customer,  the  government,  to 
establish  prices  on  production  contracts. 

In  early  production  lots,  when  the  configuration  of  the  product  may  still  be  uncer¬ 
tain  and  manufacturing  processes  are  not  finalized,  production  contracts  are  typically  cost- 
reimbursement.  Production  contracts  that  are  cost-reimbursement  are  subject  to  cost  realism 
analysis,  just  like  cost-reimbursement  development  contracts,  conducted  both  by  cost  estima¬ 
tors  and  pricers.  However,  because  production  of  major  weapon  systems  is  usually  sole  source, 
the  incentives  for  contractors  on  cost-reimbursement  production  contracts  change  because 
sole-source  producers  no  longer  have  to  worry  about  competition.  Producers  that  are  able  to 
convince  the  government  that  the  costs  to  do  the  work  are  high  will  be  able  to  establish  a  high 
starting  point  before  the  production  contracts  transition  to  fixed-price  types.  Future  negotia¬ 
tions  will  then  be  baselined  off  this  higher  starting  point.  The  government  thus  has  a  strong 
incentive  to  try  to  negotiate  lower  prices,  but  this  may  be  limited  by  the  structure  of  cost- 
reimbursable  contracts,  which  allow  for  the  reimbursement  for  all  reasonable  and  allowable 
costs. 

In  later  lots,  when  technical  and  cost  risks  have  been  lowered,  production  contracts  are 
generally  fixed-price.  It  is  on  fixed-price  production  contracts  that  government  pricing  efforts 
can  have  their  largest  effects  in  lowering  price  by  challenging  contractors’  costs  in  their  propos¬ 
als.  The  FAR  advises  that  this  is  when  should-cost  reviews  should  be  considered,  as  explained 
in  the  next  section  of  this  chapter. 

The  main  objective  of  contract  cost/price  analysis  is  to  obtain  goods  and  services  at  fair 
and  reasonable  prices  (Office  of  the  Deputy  Director  of  Defense  for  Procurement  and  Acquisi¬ 
tion  Policy  for  Cost,  Pricing,  and  Finance,  no  date).  In  the  Air  Force,  analysts  who  specialize 
in  contract  cost/price  analysis  make  up  a  small  subset  of  contracting  personnel  and  are  gener¬ 
ally  known  as  pricers.  Some  level  of  pricing  analysis  is  required  on  all  contracting  actions.  Air 
Force  pricers  typically  perform  the  pricing  function  on  large  and  complex  contracts,  while  the 
contracting  officer  typically  does  this  analysis  on  simple  or  lower- dollar-value  contracts.4 

Pricers  typically  conduct  price  and/or  cost  analyses,  analyzing  cost  element-by- element, 
to  assist  in  making  a  determination  of  a  fair  and  reasonable  price  and  negotiating  to  that  target. 
Because  data  exist  on  prior  production  lots  for  the  same  weapon  system,  technical  assessments 
are  less  important  (assuming  configurations  remain  the  same  lot  to  lot)  and  recent  historical 
cost  data  become  more  critical  in  negotiations.  Fearning-curve  analyses  are  often  used  to  esti¬ 
mate  prices  of  follow-on  production  lots. 

As  part  of  its  larger  study  of  contract  pricing  in  the  Air  Force,  the  RAND  research  team 
attempted  to  collect  information  on  the  savings  achieved  by  pricers,  defined  as  the  difference 
between  the  contractor’s  proposed  price  for  a  contract  and  the  negotiated  price. 

The  research  team  was  able  to  obtain  this  information  from  one  pricing  location.5  The 
database  from  this  location  showed  that  proposed  prices  on  contracts  where  pricers  were 
involved  were  reduced  by  percentages  averaging  in  the  high  teens  over  the  past  several  years. 


4  The  Air  Force  Materiel  Command  Federal  Acquisition  Regulation  Supplement  sets  the  threshold  for  requesting  pricing 
assistance  at  $10  million  at  the  Air  Armament  Center,  Aeronautic  Systems  Center,  and  Electronic  Systems  Center  locations 
that  have  centralized  pricing  support.  The  threshold  is  $1  million  at  the  logistics  centers. 

5  Although  this  information  is  recorded  for  individual  contract  actions,  it  apparently  is  not  routinely  organized  and  com¬ 
piled  in  a  database  and  was  available  from  only  one  pricing  office. 
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In  deciding  on  various  approaches  to  contract  pricing  and  negotiation,  one  would  like 
to  know  the  savings  achieved  by  each  approach  on  similar  procurements — that  is,  when  done 
by  the  procuring  contracting  officer,  when  done  with  pricer  support,  and  when  done  using 
should-cost  estimates.  This  ideal  comparison  is  difficult  or  even  impossible  to  achieve  in  prac¬ 
tice  because  different  approaches  are  used  in  different  circumstances,  and  the  circumstances 
probably  affect  the  potential  for  cost  savings.  The  ideal  comparison  would  hold  constant  such 
key  conditions  as  the  contractor,  product  or  weapon  system  procured,  and  type  of  contract, 
and  then  compare  outcomes  using  different  pricing  and  negotiation  approaches. 

The  next  section  of  this  chapter  provides  background  on  should-cost  reviews  as  defined 
in  the  FAR. 


Background  on  Should-Cost  Reviews  from  the  Federal  Acquisition 
Regulation 

It  is  important  to  understand  that  the  FAR  provides  a  definition  of  should-cost  review  that  is 
narrower  than  the  way  the  phrase  is  used  in  guidance  from  USD(AT&L)  in  2010  and  2011 
(Carter,  2010a,  2010b;  Carter  and  Hale,  2011)  and  the  Air  Force  guidance  in  2011  (Morin  and 
Van  Buren,  2011).  The  FAR  provides  guidance  for  the  contracting  function,  and  the  should- 
cost  reviews  described  in  the  FAR  are  a  specialized  form  of  cost  analysis  used  to  identify  inef¬ 
ficiencies  and  lower  costs  in  support  of  contract  negotiations.  Because  the  recent  USD(AT&L) 
and  Air  Force  guidance  also  have  the  objective  of  reducing  cost,  the  FAR  definition  is  relevant 
here.  However,  different  kinds  of  should-cost  reviews  can  be  and  have  been  used  for  other 
purposes.  For  example,  they  can  be  initiated  and  used  by  companies  seeking  to  improve  their 
own  operations.  Additional  examples  are  provided  in  Chapter  Four,  on  the  use  of  should-cost 
reviews  in  commercial  organizations.  DoD  may  use  different  kinds  of  should-cost  reviews  to 
obtain  better  understanding  of  the  costs  and  status  of  programs  rather  than  in  support  of  a 
current  contract  negotiation. 

What  Is  a  Should-Cost  Review? 

Subpart  15.4  of  the  FAR  provides  the  policies  and  procedures  for  the  costing  and  pricing  of 
proposed  contracts.  The  ultimate  goal  of  contract  cost/price  analysis  is  to  arrive  at  a  fair  and 
reasonable  price  to  be  paid  by  the  government  to  a  contractor.  Should-cost  reviews  are  autho¬ 
rized  by  FAR  15.407-4;  however,  the  regulation  only  broadly  defines  the  ground  rules  and 
purposes  of  a  should-cost  review,  identifies  when  a  should-cost  review  should  be  conducted 
and  the  single  product  it  is  to  provide,  and  outlines  the  personnel  and  organizational  structure 
required. 

According  to  FAR  15.407-4,  a  should-cost  review  is  a  specialized  form  of  contract  cost 
analysis.  Its  broad  purpose  is  to  lower  the  cost  of  production  contracts  by  identifying,  and 
establishing  plans  to  eliminate,  inefficiencies  at  contractor  facilities.  Its  narrow  purpose  is  to 
write  a  report  about  those  inefficiencies  that  can  be  used  to  help  determine  the  government’s 
target  price  in  contract  negotiations. 

Although  a  should-cost  review  does  not  substitute  for  price  or  cost  analysis,  it  can  be  used 
to  support  proposal  analysis,  specifically  to  help  the  government  “develop  realistic  objectives 
for  negotiation”  (FAR  15.407-4).  Should-cost  reviews  can  be  small  or  large:  They  can  target 
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specific  concerns  or  examine  the  entire  production  process.  No  specific  timeline  for  completion 
of  should-cost  reviews  is  provided. 

FAR  15.407-4  specifies  two  distinct  but  complementary  types  of  should-cost  reviews: 
program  and  overhead.  Program  should-cost  reviews  examine  direct  labor  hours  and  material 
dollars,  tooling  and  facilities,  labor  skill  mix,  and  production  processes.  Overhead  should-cost 
reviews  examine  rates  and  all  indirect  charges  to  a  contract,  including  profit.  Program  and 
overhead  should-cost  reviews  can  be  conducted  in  tandem  or  separately,  but  the  skills  and 
experiences  necessary  to  conduct  them  can  differ  substantially. 

When  to  Conduct  a  Should-Cost  Review 

FAR  15.407-4  indicates  that  a  program  should-cost  review  should  be  considered,  especially  for 
major  acquisition  programs,  when 

•  some  production  has  already  taken  place  and  there  are  requirements  for  a  substantial 
amount  of  future  production 

•  costs  are  increasing 

•  the  work  (i.e.,  product)  is  well-defined  enough  to  allow  effective  analysis  and  major 
changes  are  unlikely 

•  adequate  time  and  personnel  are  available  to  conduct  the  review. 

Overhead  should-cost  reviews  should  be  considered  for  contractor  sites  that 

•  produce  a  major  system 

•  have  a  high  dollar  amount  of  government  contracts  or  a  large  number  of  proposals 

•  have  a  high  level  of  government  participation  or  noncompetitive  contracts 

•  have  been  subject  to  changes  in  corporate  organization,  management,  or  accounting 
systems. 

Should-Cost  Review  Analytical  Methods 

Like  standard  contract  cost  and  price  analyses,  should-cost  reviews  can  utilize  a  variety  of 
methods,  but,  unlike  standard  cost  or  price  analyses,  should-cost  reviews  help  estimate  the 
cost  of  production  assuming  the  most  efficient  existing  production  processes  and  technologies. 
Actual  costs  that  embody  historical  or  contractor-specific  inefficiencies  are  to  be  disregarded  as 
the  basis  for  future  costs. 

FAR  15.407-4  does  not  require,  recommend,  or  even  state  specific  analysis  methods 
(such  as  value  engineering,  industrial  engineering,  review  of  earned  value  metrics,  or  cost- 
improvement  curve  analysis).  In  Air  Force  experience,  a  variety  of  analytical  methods  have 
been  used,  as  discussed  further  in  Chapter  Three. 

Should-Cost  Team  Composition 

The  FAR  does  not  specify  what  organization  or  function  should  lead  the  should-cost  team,  nor 
does  it  specify  required  team  members.  However,  FAR  15.407-4  does  recommend  a  “multi¬ 
functional  team”  of  contracting,  administration,  pricing,  and  engineering  personnel.  For  a 
program  should-cost  review,  the  contracting  officer  is  assigned  with  identifying  “elements  of 
the  contractor’s  operation”  with  highest  likely  cost  savings  and  matching  available  personnel 
against  those  areas.  The  FAR  does  not  indicate  which  organizations  are  to  provide  these  per- 
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sonnel.  Similarly,  no  personnel  management  responsibility  is  explicitly  assigned  for  overhead 
should-cost  reviews. 

Should-Cost  Reports 

At  the  completion  of  its  analyses,  the  should-cost  team  summarizes  its  findings  in  a  report.  For 
a  program  should-cost  review  supporting  a  contract  proposal,  the  should-cost  team  must  write 
up  findings  and  recommendations,  which  the  program’s  contracting  officer  must  take  into 
account  during  contract  negotiations.  The  contracting  officer  must  then  provide  the  admin¬ 
istrative  contracting  officer  (ACO)  a  report  of  all  identified  inefficiencies  and  how  they  were 
corrected  or  dispositioned,  and  establish  a  monitoring  plan. 

Overhead  should-cost  review  teams  must  also  write  up  findings;  however,  this  report 
must  be  provided  to  the  ACO,  who  must  use  it  when  negotiating  forward  pricing  rate  agree¬ 
ments.  The  ACO  must  establish  a  monitoring  plan. 


Latest  Guidance  on  Should-Cost  Reviews  Supplemental  to  the  Federal 
Acquisition  Regulation 

The  Defense  Federal  Acquisition  Regulation  Supplement  and  its  complementary  Procedures, 
Guidance,  and  Information  (PCI)  resource  provide  additional  guidance  for  implementing  the 
FAR  within  DoD  (Defense  Procurement  and  Acquisition  Policy,  2011).  The  guidance  reflects 
laws,  regulations,  and  policies.  In  the  PCI  that  addresses  contract  pricing,  updated  May  18, 
2011,  the  guidance  dramatically  broadens  the  criteria  for  conducting  a  program  should-cost 
review,  consistent  with  USD(AT&T)  guidance:  “DoD  contracting  activities  should  consider 
performing  a  program  should-cost  review  before  award  of  a  definitive  contract  for  a  major 
system  .  . .”  (Defense  Procurement  and  Acquisition  Policy,  2011,  Subpart  215.4).  Major  systems 
are  those  with  estimated  research  and  development  costs  greater  than  $140  million  in  FY  2000 
constant  year  dollars  or  procurement  costs  greater  than  $660  million  in  FY  2000  constant 
dollars. 


CHAPTER  THREE 


What  Has  Been  the  Air  Force  Experience  with  Should-Cost 
Reviews? 


This  chapter  presents  the  findings  on  the  first  of  two  key  research  questions  addressed  in  this 
report:  What  has  been  the  Air  Force  experience  with  should-cost  reviews? 

The  chapter  provides  summaries  of  experiences  with  should-cost  reviews  and  lessons 
learned  in  a  few  key  areas,  derived  from  a  review  of  literature  on  DoD  should-cost  reviews  and 
from  interviews  with  participants  in  prominent  should-cost  reviews  of  Air  Force  programs. 


Sources  of  Information  on  Air  Force  Experience  with  Should-Cost  Reviews 

The  next  sections  address  Air  Force  experience  with  should-cost  reviews.  Because  there  is  no 
dedicated  organization  within  the  Air  Force  with  responsibility  for  should-cost  reviews  and  no 
dedicated  place  where  information  regarding  them  is  retained,  the  study  team  had  to  gather 
information  on  historical  experience  from  disparate  sources.  The  experience  was  summarized 
from  interviews  with  participants  in  Air  Force  should-cost  reviews  over  roughly  the  past  20 
years,  from  the  little  written  documentation  of  the  reviews  that  could  be  obtained,  and  from  a 
literature  review.  The  sources  cover  two  time  periods.  The  literature  reviewed  here  dates  from 
1970  through  1988  and  spans  an  era  when  the  reviews  were  conducted  more  often  than  they 
have  been  subsequently.  Little,  if  any,  relevant  literature  was  found  after  this  time  period. 
The  interviews  and  written  documentation  encompass  experience  from  the  late  1980s  through 
2011. 

We  stress  that  many  of  the  reviews  discussed  during  interviews  and  found  in  the  literature 
do  not  meet  the  FAR  definition  of  a  should-cost  review,  even  though  they  are  generally  referred 
to  as  such.  Some  of  the  reviews  are  reactive  responses  to  troubled  programs  rather  than  proac¬ 
tive  analyses  done  as  part  of  contract  negotiations.  We  include  this  broad  experience  because 
similar  prominent  reviews  are  being  done  today,  and  the  historical  experience  with  reviews  that 
do  not  meet  the  FAR  definition  may  be  helpful  for  similar  reviews  today. 

We  reviewed  the  literature  to  find  some  historical  context  for  the  Air  Force’s  experience 
with  should-cost  reviews.  We  sought  to  understand  when  and  why  the  Air  Force  began  to 
use  should-cost  reviews.  Then  we  focused  on  three  aspects  of  the  Air  Force’s  experience  with 
should-cost  reviews  that  were  most  likely  to  be  of  interest  to  decisionmakers  in  enhancing  the 
Air  Force’s  current  capability  to  conduct  such  reviews. 

First,  we  were  interested  in  the  results.  In  particular,  do  should-cost  reviews  save  money? 
There  are  various  ways  to  measure  savings.  Savings  on  a  sole-source  contract  can  be  calculated 
as  a  reduction  from  the  price  proposed  by  the  contractor.  But  because  the  Air  Force  routinely 
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negotiates  and  achieves  savings  from  proposed  prices  on  sole-source  contracts,  a  measure  of 
savings  that  is  of  greater  interest  is  a  comparison  of  the  savings  achieved  through  the  use  of 
should-cost  reviews  to  the  savings  achieved  by  other  techniques,  such  as  by  a  pricer  using  con¬ 
ventional  forms  of  analysis. 

Second,  we  were  interested  in  the  composition  of  the  teams  that  conducted  should-cost 
reviews.  In  particular,  we  wanted  to  determine  the  typical  size  of  the  teams,  what  functional 
areas  of  expertise  were  used,  and  the  typical  duration  of  the  reviews. 

Third,  we  were  interested  in  the  organizational  support  needed  for  the  reviews  and  pro¬ 
cesses  used  in  the  reviews,  especially  any  specialized  training  that  was  used  or  needed. 


Literature  Review  Results  of  Air  Force  Experience  with  Should-Cost  Reviews 

Historical  Context 

The  Air  Force  has  had  the  capability  to  conduct  should-cost  reviews  for  at  least  50  years.  The 
Air  Force  used  teams  to  assess  the  organization  and  management  of  its  contractors  in  the 
1960s.  The  first  of  these  assessments,  which  were  called  Industrial  Management  Assistance 
Surveys,  was  done  in  I960  and  was  described  as  somewhat  comparable  to  a  should-cost  review. 
The  Air  Force  conducted  many  surveys  of  smaller  scope  during  the  1960s,  although  they  were 
advisory  in  nature  and  not  intended  for  use  in  contract  negotiations  (U.S.  General  Accounting 
Office  [GAO],  1970). 

The  GAO  found  that  although  should-cost  concepts  were  recognized  in  DoD  policy,  the 
analyses  were  rarely  used,  and  historical  costs  were  usually  used  as  the  basis  for  contract  nego¬ 
tiations  (GAO,  1970,  p.  2).  DoD  officials  at  the  time  expressed  the  belief  that  contracting  situa¬ 
tions  that  required  a  should-cost  review  were  rare  and  that  if  circumstances  arose  that  required 
such  a  review,  an  ad  hoc  team  would  be  preferable  to  establishing  a  permanent  capability  to  do 
them  (GAO,  1970,  p.  13). 

One  of  the  big  initiatives  in  OSD  to  improve  cost  estimating  during  this  time  was  to 
require  DoD  contractors  to  report  their  costs  in  a  standardized  format  and  use  the  histori¬ 
cal  costs  in  the  standard  format  as  the  basis  for  estimating  the  costs  of  similar  future  weapon 
systems.  Advocates  of  this  approach  urged  the  use  of  competitive  bids  for  fixed-price  contracts 
as  the  best  way  to  obtain  low  costs  and  better  performance.  An  argument  against  the  use  of 
should-cost  analysis  was  the  large  amount  of  data  required  (Fitzgerald,  1972). 

In  1965,  the  Air  Force  awarded  a  fixed-price  contract  for  development  and  procurement 
of  the  C-5A  cargo  aircraft,  reflecting  the  policy  preference  for  fixed-price  contracts  as  the  way 
to  obtain  low  costs  and  better  performance.  The  program  experienced  large  cost  overruns. 
Air  Force  Deputy  for  Management  Systems  A.  Ernest  Fitzgerald,  on  the  staff  of  the  Assistant 
Secretary  of  the  Air  Force  for  Financial  Management,  had  looked  into  the  problems  on  the 
program  and  was  asked  in  1968  to  testify  before  the  Joint  Economic  Committee  of  Congress.1 


1  Fitzgerald  is  an  industrial  engineer  by  training  who  headed  a  small  consulting  firm  that  conducted  should-cost  analyses 
prior  to  his  working  for  the  Air  Force.  He  was  hired  as  a  high-level  civil  servant  by  the  Air  Force  in  1965.  His  testimony 
embarrassed  Air  Force  officials,  Fitzgerald’s  civil  service  tenure  was  revoked,  and  within  a  year  the  Air  Force  reorganized 
Fitzgerald  out  of  a  job.  He  was  reinstated  a  few  years  later  after  lengthy  litigation.  Fitzgerald  believed  that  the  historical 
costs  used  to  estimate  costs  of  future  similar  systems  contained  waste  that  was  perpetuated  by  using  the  historical  costs  as 
the  basis  for  estimating  future  programs.  He  advocated  estimating  costs  using  bottom-up  or  engineering  approaches  that 
would  arrive  at  what  the  product  should  cost  if  produced  efficiently  (Fitzgerald,  1972,  pp.  26-28,  34-42). 
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After  Fitzgerald’s  testimony  to  Congress  about  cost  overruns  on  the  fixed-price  C-5A 
contract,  in  1969  the  Joint  Economic  Committee’s  Subcommittee  on  Economy  in  Govern¬ 
ment  asked  the  GAO  to  explore  the  feasibility  of  using  should-cost  ideas  in  its  assessments  of 
contractor  performance. 

At  the  time  of  the  1970  GAO  report,  the  GAO  was  conducting  four  trial  should-cost 
reviews  to  assess  what  problems  and  benefits  might  be  expected,  and  other  issues.  In  1971, 
the  GAO  reported  on  the  results  of  its  trial  reviews.  It  found  that  should-cost  reviews  can  be 
very  beneficial  and  that  procuring  agencies  that  conducted  the  reviews  prior  to  contract  award 
would  allow  their  negotiators  to  bargain  from  a  stronger  position  (GAO,  1971).  The  report 
noted  that  the  Air  Force  had  started  its  first  should-cost  review  (p.  19).  By  the  early  1970s, 
should-cost  reviews  were  used  more  widely  within  DoD. 

Do  Should-Cost  Reviews  Save  Money? 

We  found  two  studies  that  made  a  rigorous  attempt  to  answer  the  question  of  whether  should- 
cost  reviews  save  money,  and  one  early  and  less  rigorous  assessment  by  GAO.  The  studies 
compared  the  savings  achieved  by  should-cost  reviews  with  the  savings  achieved  using  other 
contract  negotiation  methods. 

The  GAO  assessed  five  should-cost  analyses  conducted  by  the  Air  Force  between  the  fall 
of  1970  and  January  1973.  The  GAO  compared  the  price  reductions  achieved  in  negotiations 
using  should-cost  reviews  on  five  contracts  with  the  reductions  achieved  on  the  same  programs 
using  other  negotiation  methods.  The  GAO  found  that  should-cost  analysis  resulted  in  higher 
savings  in  four  of  the  five  cases.  The  GAO  specified  the  difference,  of  7  percent,  on  only  one 
of  the  contracts.  The  GAO  also  added  the  caveat  that  the  negotiated  amounts  did  not  neces¬ 
sarily  result  in  savings  to  the  government  because  four  of  the  contracts  were  fixed-price  incen¬ 
tive  (FPI)  contracts  on  which  the  final  price  would  not  be  determined  until  the  contract  was 
complete  (GAO,  1973). 

The  objective  of  the  second  study  was  to  compare  cost  outcomes  on  contracts  negotiated 
with  the  use  of  should-cost  reviews  to  contracts  negotiated  without  such  reviews  (Schaefer 
and  Birkhead,  1975).  The  study’s  sample  began  with  17  FPI  contracts  for  which  should-cost 
reviews  were  used.  FPI  contracts  were  selected  because,  unlike  firm  fixed-price  contracts,  the 
FPI  contracts  were  subject  to  a  redetermination  of  costs  after  the  contract  was  completed  so 
that  the  government  had  access  to  the  final  cost  data.  This  restriction  was  necessary  because  the 
authors  wished  to  measure  cost  savings  at  five  stages  in  the  contracting  process  that  included 
the  potential  for  contract  modifications  to  be  used  to  increase  contract  costs,  and  thus  final 
contract  costs  were  required.  Of  the  17  contracts,  only  four  had  final  cost  data  available.  For 
comparison,  the  study  authors  selected  four  contracts  with  the  same  manufacturer  and  for  the 
same  product,  but  negotiated  with  methods  other  than  should-cost. 

The  five  stages  that  were  compared  were 

1.  proposed  value  to  negotiated  value 

2.  proposed  value  to  completed  value 

3.  revised  (by  contract  modification)  value  to  completed  value 

4.  cost  of  revision  as  a  percent  of  proposed  value 

5.  cost  of  revision  as  a  percent  of  negotiated  value. 
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The  authors  found  no  significant  difference  in  cost  outcomes  in  any  of  the  five  stages 
(Schaefer  and  Birkhead,  1975,  p.  56).  The  extremely  small  sample  size,  however,  limits  the 
conclusions  that  can  be  drawn  from  the  study. 

The  third  study  we  found  that  attempted  a  rigorous  assessment  of  should-cost  results  was 
similar  in  methodology  to  the  second  study  and  analyzed  matched  pairs  of  contracts  for  the 
same  weapon  system  and  contractor,  with  one  contract  using  should-cost  and  the  other  using  a 
different  negotiating  method  (Conway  and  Howenstine,  1983).  The  authors  stressed  that  their 
data  were  based  on  a  convenience  sample  of  available  data,  used  because  of  a  lack  of  historical 
contract  data.  Then  as  now,  the  Air  Force  had  not  established  a  central  repository  for  data  on 
should-cost  contracts,  and  in  many  cases  the  organizations  queried  did  not  maintain  or  were 
reluctant  to  search  for  the  type  of  data  needed  to  perform  their  analysis  (Conway  and  Howen¬ 
stine,  1983,  p.  28). 

The  authors  compared  the  percentage  change  in  negotiated  price  from  the  proposed  price 
and  found  no  statistically  significant  difference  in  the  eight  matched  pairs  of  Air  Force  con¬ 
tracts  (Conway  and  Howenstine,  1983,  p.  43).  The  authors  urged  caution  in  interpreting  the 
results  due  to  the  small  sample  size. 

A  less-controlled  assessment  was  made  by  A.  Ernest  Fitzgerald.  He  found  that  the  success 
of  the  early  should-cost  analyses  and  other  cost-control  efforts  in  reducing  cost  to  the  govern¬ 
ment  was  mixed  at  best.  Fitzgerald  wrote  that,  as  a  consultant  conducting  should-cost  reviews 
of  large  weapon  system  programs  in  the  1960s,  there  was  never  difficulty  in  finding  places  to 
save  money.  “All  of  them  were  as  fat  as  lard  hogs,”  he  writes;  the  difficulty  was  in  getting  action 
(Fitzgerald,  1972,  p.  6).  In  1972,  by  which  time  DoD  had  adopted  the  use  of  should-cost  pric¬ 
ing,  Fitzgerald  argued  that  the  approach  had  changed  to  “subjective,  qualitative  reviews”  that 
saved  very  little  money.  Even  when  potential  savings  were  identified,  they  were  seldom  realized 
because  they  were  not  negotiated  or  were  given  back  to  the  contractor  in  subsequent  contract 
changes  (Fitzgerald,  1972,  p.  363). 

What  Is  the  Composition  of  Should-Cost  Teams? 

The  second  issue  we  wished  to  investigate  in  the  literature  review  was  the  composition  of 
should-cost  teams. 

Fitzgerald  (1972)  indicated  that  early  should-cost  activities  conducted  on  aerospace  pro¬ 
grams  utilized  a  small,  highly  trained  cadre  of  industrial  engineers  who  compared  the  effi¬ 
ciency  of  production  processes  at  contractor  plants  with  industry  best  standards.  For  analyses 
requested  by  and  conducted  for  the  manufacturer,  no  contracting  or  pricing  team  members 
were  needed. 

After  reinstatement  to  his  Air  Force  job  in  the  early  1970s,  Fitzgerald  continued  to  be 
involved  in  should-cost  reviews  and  other  cost-control  efforts  until  his  retirement  in  2006.  In 
an  interview  conducted  in  2009,  he  continued  to  recommend  work-measurement  techniques 
for  should-cost  reviews  today:  benchmarking  actual  hours  versus  standard  work  hours  for  each 
production  process  and  using  the  resulting  information  to  determine  which  areas  need  the 
most  focus.2 

The  Air  Force  had  a  Should  Cost  pamphlet,  designated  Air  Force  Pamphlet  (AFP)  70-5 
and  now  obsolete,  that  provided  guidance  on  the  use  of  should-cost  reviews,  including  guid- 


Interview  with  A.  Ernest  Fitzgerald,  retired  Air  Force  Financial  Managament  and  Comptroller  staff  member,  2009. 
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ance  regarding  composition  of  the  teams  (Headquarters  U.S.  Air  Force,  1989).  The  guidance 
called  for  integrated  teams  formed  of  government  acquisition,  contracting,  contract  adminis¬ 
tration,  pricing,  auditing,  and  technical  representatives.  According  to  the  pamphlet,  the  team 
should  be  composed  of  highly  skilled,  dedicated  people  available  for  full-time  duty;  a  mix  of 
people  familiar  with  the  contractor’s  operations  and  outside  experts  is  desirable;  and  the  team 
should  be  headed  by  a  chief  at  the  GM-15/colonel  rank  or  higher  (p.  4).  The  pamphlet  states 
(p.  5)  that  the  procuring  organization  should  be  strongly  represented,  especially  by  the  procur¬ 
ing  contracting  officer,  technical  experts  in  engineering,  manufacturing,  logistics,  and  other 
technical  areas  as  needed,  and  one  or  more  price  analysts;  and  that  the  principal  negotiator  and 
the  contracting  officer  or  a  designee  should  be  full-time  members. 

The  Army’s  1972  Procurement:  Should-Cost  Analysis  Guide  cites  teams  composed  of  spe¬ 
cialists  in  engineering,  pricing,  audit,  procurement,  and  management  (U.S.  Army  Materiel 
Command  Headquarters,  1972,  p.  1-1).  In  addition  to  technical  skills,  the  guide  advises  that 
good  skills  in  oral  and  written  communication,  including  the  ability  to  communicate  across 
disciplines,  are  important.  The  guide  advises  that  the  single  most  important  attribute  in  team 
members  is  high  motivation,  which  is  required  because  of  the  long  work  hours  and  absence 
from  home  and  family  (p.  3-3).  The  guide  also  advises  that  the  number  of  members  on  the 
team  and  the  mix  of  skills  depend  on  the  nature  of  the  task  (p.  3-4). 

Weida  and  Sloan  (1972)  analyzed  the  participation  of  specialists  from  the  Air  Force  plant 
representative  offices  in  Air  Force  should-cost  studies  and  found  that  industrial  engineers  were 
the  most  utilized  specialty,  followed  by  pricers.  Other  specialists  included  development  engi¬ 
neers,  contract  specialists,  and  quality  engineers  (pp.  99-100). 

Ulrich  (1973)  also  found  that  the  two  most  commonly  desired  functional  specialties  on 
should-cost  teams  were  industrial  engineers  and  price  analysts  (p.  85).  He  noted  that  inad¬ 
equate  writing  skills  have  plagued  should-cost  teams,  and  in  particular  that  the  inability  of 
the  teams  to  support  their  position  in  writing  is  a  serious  handicap  to  the  contract  negotiator 
(p.  99). 

Williams  (1985)  found  that  teams  ranged  in  size  from  eight  to  80  members  and  that  the 
duration  of  the  reviews  ranged  from  a  few  weeks  to  several  months  (p.  24). 

What  Was  the  Organizational  Support  and  Process  for  Conducting  Reviews? 

The  third  issue  we  wished  to  focus  on  in  the  literature  search  was  the  organizational  support 
required  for  should-cost  reviews  and  the  processes  for  conducting  the  reviews,  especially  any 
specialized  training  that  was  used  or  needed. 

Weida  and  Sloan  (1972)  considered  the  establishment  of  a  permanent  organization  within 
the  Air  Force  to  support  should-costs.  They  argued  that  a  permanent  organization  is  desirable 
only  if  top-level  management  supports  the  concept  of  a  should-cost  capability  (p.  124).  The 
two  main  areas  of  support  needed  would  be  additional  manpower  and  additional  training  in 
the  concepts  and  analytical  methods  of  a  should-cost  study.  They  noted  the  need  for  a  train¬ 
ing  and  resource  center  that  would  maintain  a  library  of  reference  material,  provide  consultant 
services,  prepare  and  maintain  a  should-cost  guide,  and  provide  training  using  lessons  learned 
from  previous  studies  (pp.  126-128). 

Regarding  the  process  of  conducting  the  reviews,  Heitman  and  King  (1984,  p.  97)  found 
a  need  for  more  guidance  for  planning  the  reviews.  They  found  that  the  sources  of  information 
most  helpful  in  the  planning  process  were  (1)  information  pertaining  to  the  contractor,  (2)  pre- 
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vious  should-cost  reports,  (3)  previous  experience,  and  (4)  AFP  70-5  (the  Air  Force  guidance 
on  should-cost  reviews,  which  is  no  longer  published). 

Williams  (1985)  wrote  that  “Research  indicates  that  the  vast  majority  of  problems  that  are 
encountered  during  a  Should  Cost  analysis  are  due  to  insufficient  planning,  insufficient  time 
to  properly  plan  the  evolution,  or  failure  to  include  the  contractor’s  input”  as  to  what  is  needed 
for  a  successful  analysis  into  the  plans  (p.  24).  Williams  stressed  the  need  for  an  adequate 
number  of  highly  skilled  personnel  to  conduct  the  analysis  and  wrote  that  planning  includes 
contact  with  the  contractor  to  determine  what  resources  are  needed  to  conduct  the  analysis. 
He  also  emphasized  that  should-cost  reviews  are  only  a  tool  that  can  be  useful  in  the  budget¬ 
ing  or  contract  negotiating  process.  The  reviews  produce  estimates  given  certain  assumptions 
and  conditions,  but  if  the  assumptions  or  conditions  change  or  are  not  achieved,  the  estimates 
will  no  longer  be  valid.  He  warned  that  should-cost  analyses  are  not  a  panacea  to  the  problem 
of  cost  overruns  (p.  25). 

To  make  effective  use  of  the  should-cost  review,  Williams  argued  that  senior  officials 
must  be  involved;  that  the  team  leader  should  be  given  a  written  charter  that  identifies  the  task, 
authority,  etc.;  and  that  the  chief  contract  negotiator  must  have  in-depth  knowledge  of  the 
should-cost  report.  For  this  reason,  the  team  leader  has  traditionally  been  the  chief  negotiator 

(PP.  44,  45,  55). 


Interview  Results  of  Air  Force  Experience  with  Should-Cost  Reviews 

In  addition  to  conducting  a  literature  review  on  the  subject  of  should-cost  reviews,  we  con¬ 
ducted  interviews  with  participants  in  Air  Force  should-cost  reviews.  The  interviews  cov¬ 
ered  reviews  conducted  from  the  late  1980s  through  2011.  In  many  cases,  analyses  that  were 
described  as  should-cost  reviews  did  not  meet  the  FAR  definition.  For  example,  some  of  the 
analyses  have  been  conducted  on  ongoing  development  programs,  whereas  the  FAR  guidance 
is  to  consider  should-cost  reviews  for  production  programs  with  well-defined  technologies  and 
designs,  when  production  processes  are  set.  Furthermore,  in  the  cases  of  analyses  of  develop¬ 
ment  programs,  the  contract  had  already  been  signed,  so  that  review  results  could  at  best  be 
used  to  inform  negotiations  on  contract  modifications.3 

Few  written  records  of  these  more  recent  reviews  were  found,  so  that  in  most  cases  we  had 
to  rely  solely  on  the  recollection  of  the  participants.  The  following  section  describes  the  goals, 
organization,  and  results  of  these  reviews. 

1980s:  B-2 

In  the  mid-to-late  1980s,  an  overhead  should-cost  review  was  conducted  for  the  B-2  bomber 
program.  Sponsored  by  the  commanding  general  of  Air  Force  Systems  Command,  predeces¬ 
sor  of  part  of  the  current  Air  Force  Materiel  Command,  it  was  focused  on  cost  control.  The 
primary  goal  was  a  better  understanding  of  contractor  overhead.  The  key  deliverable  was  a  new 
forward  pricing  rate  agreement  at  Northrop,  the  prime  contractor.  The  review  was  performed 
onsite  at  Northrop  in  Southern  California,  at  Boeing  in  Seattle,  and  at  Vought  in  Texas. 


3  Some  participants  believe  that  should-cost  reviews  can  be  useful  during  development  to  reveal  hidden  assumptions 
underlying  the  program  and  to  provide  insight  into  the  potential  for  future  cost  growth  or  technical  challenges. 
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The  review  was  complicated  by  the  classification  of  the  program.  Anywhere  from  80  to 
100  people  were  on  the  overall  team,  which  was  led  by  an  Air  Force  colonel.  A  few  subteams 
formed,  each  working  for  several  weeks  at  a  time.  The  overall  review  required  16  weeks,  includ¬ 
ing  writing  the  reports  and  negotiations. 

Many  costs  of  doing  business  were  impossible  to  reduce  (e.g.,  an  extremely  large  number 
of  personnel  in  security).  A  senior  government  executive  successfully  negotiated  the  contractor 
overhead  rates  down.  Despite  these  negotiations,  the  rates  eventually  rose  slightly  higher  than 
the  contractor’s  original  position.  Our  interviewee  believed  that  this  was  a  less-than-successful 
should-cost  review.4 

1990s:  F-22 

A  should-cost  review  of  the  F-22  program  was  conducted  in  the  early-to-mid  1990s,  analyz¬ 
ing  both  development  and  planned  production  (production  would  not  start  until  2000).  Three 
teams  went  out  to  major  contractor  locations — Fort  Worth,  Texas;  Marietta,  Georgia;  and 
Seattle,  Washington.  The  engine  program  was  not  reviewed. 

The  should-cost  review  addressed  consolidation  issues  raised  by  the  purchase  by  Lock¬ 
heed  of  General  Dynamics’  Fort  Worth  plant.  The  government  believed  that  consolidation  of 
personnel  should  have  driven  considerable  improvements  in  costs.  However,  Lockheed  was 
reluctant  to  reduce  employment  despite  the  consolidation,  resulting  in  little  cost  reduction. 

Costs  were  adequately  represented  and  understood  on  both  sides.  While  the  review  gener¬ 
ated  considerable  technical  understanding,  better  cost  estimates,  and  a  higher  degree  of  con¬ 
fidence,  it  did  not  result  in  lower  costs  on  the  F-22  program.  Interviewees  did  not  believe  the 
government  review  drove  contractor  behavior  to  reduce  cost  or  cost  growth.5 

1990s:  C-17 

A  sequence  of  should-cost  reviews  was  conducted  on  the  C-17  program  in  the  1990s.  The  review 
conducted  in  1994 — described  as  the  “celebrity  should-cost”  by  one  interviewee  because  so 
many  high-level  officials  were  involved — received  the  highest  level  of  attention  inside  the  Air 
Force.  Some  analysts  considered  it  the  most  successful  should-cost  review  of  those  noted  here. 

Development  and  procurement  costs  had  risen  dramatically  on  the  C-17  program  from 
baseline  estimates,  and  average  unit  procurement  costs  of  early  production  lots  remained  high 
in  the  early  1990s.  The  purpose  of  the  1994  review  was  to  save  the  program  from  being  can¬ 
celled.  USD(AT&L)  had  indicated  that  unless  the  program  was  both  technically  and  cost  sta¬ 
bilized,  no  more  than  40  aircraft  would  likely  be  bought.  Making  this  threat  credible,  the  gov¬ 
ernment  was  considering  a  modified  Boeing  747  and  a  C-5  variant  as  alternatives.  Hence,  the 
contractor,  McDonnell-Douglas,  was  motivated,  as  was  the  Air  Force.  A  fully  integrated  con¬ 
tractor  and  government  team  was  formed,  with  top-level  corporate  and  Air  Force  leadership. 
The  should-cost  review  team  lead  was  the  government  program  director,  an  Air  Force  general 
officer.  However,  the  review  team’s  day-to-day  operations  were  run  by  an  Air  Force  colonel. 

About  40  people  were  on  the  team.  The  review  lasted  from  12  to  16  weeks,  and  subteams 
rotated  in  and  out  of  locations  periodically.  A  dozen  full-time  core  personnel  spent  six  to  eight 
weeks  at  Long  Beach,  with  the  other  personnel  spread  out  over  the  duration  of  the  review.  They 


4  Interview  with  Defense  Contract  Management  Agency  staff  member,  2009  (name  withheld  by  request). 

5  Interview  with  Defense  Contract  Management  Agency  staff  member,  2009  (name  withheld  by  request). 
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examined  touch  labor,  engineering,  and  overhead.  The  contractor  provided  subteam  leaders  to 
the  review  team,  providing  excellent  exposure  at  the  working  level. 

In  a  cooperative  effort  with  the  company,  the  review  team  developed  a  consensus  cost 
position  for  the  follow-on  40  units.  The  team  identified  significant  cost  reductions  in  both 
overhead  and  technical  areas. 

The  overhead  review  team  found  economies  of  scale  in  purchasing  and  found  substantial 
cost  savings  in  schedule  stability  and  a  multiyear  buy.  A  small,  experienced  team  worked  the 
schedule  and  production  rates.  Most  other  issues  (e.g.,  the  number  of  engineers  in  overhead, 
marketing  and  process  studies)  were  left  for  management  and  program  personnel  to  negotiate. 

The  C-17  program  continued  production,  with  more  than  200  aircraft  delivered  by  Sep¬ 
tember  2010.  The  average  procurement  unit  cost  decreased  significantly  in  later  years,  in  large 
part  due  to  the  economies  achieved  with  annual  production  rates  higher  than  the  six  aircraft 
per  year  bought  in  1994. 

1990s:  T-6  Joint  Primary  Aircraft  Training  System 

The  Air  Force  began  low  rate  initial  production  of  the  Joint  Primary  Aircraft  Training  System 
in  1995.  It  appears  that  the  prime  contractor,  Raytheon,  had  counted  on  additional  sales  in  a 
robust  international  market  to  lower  unit  costs  of  the  trainer  aircraft.  However,  there  had  been 
no  international  sales,  and  Raytheon’s  costs  of  production  were  higher  than  the  price  the  gov¬ 
ernment  was  paying.  Because  the  existing  contract  was  firm  fixed-price,  this  led  to  large  losses. 

In  1995-1996,  a  small  overhead  should-cost  review  team  was  formed  comprising  10-12 
people  for  a  six-week  core  period,  and  other  personnel  for  a  six-month  period.  The  team  was 
sourced  from  the  Air  Force,  the  Defense  Contract  Management  Agency  (DCMA),  and  the 
Defense  Contract  Audit  Agency. 

Review  participants  had  mixed  opinions  regarding  the  outcome  of  the  review.  One  par¬ 
ticipant  believed  the  review  did  not  lower  costs  to  the  government  but  did  result  in  better 
understanding  and  insight  of  the  competitive  situation,  and  that  the  Air  Force  used  the  review 
to  inform  its  acquisition  strategy  for  the  follow-on  contract.6  (As  noted,  Raytheon  lost  money 
on  the  initial  firm  fixed-price  contract,  and  the  unit  cost  to  the  government  of  the  T-6  increased 
on  the  subsequent  production  contract.)  Another  participant  believed  that  the  review  resulted 
in  lower  production  costs  than  were  originally  proposed  for  the  follow-on  contract.7 

2000s:  Global  Hawk,  Evolved  Expendable  Launch  Vehicle,  and  Global  Hawk  Again 

In  2002-2003,  the  Air  Force  conducted  an  overhead  and  program  should-cost  review  of  the 
Global  Hawk  program.  Some  critics  believed  that  the  unmanned  aircraft  should  be  as  inex¬ 
pensive  as  the  less-capable  Defense  Advanced  Research  Projects  Agency  prototype,  which  was 
delivered  for  $10  million.  The  review  teams  examined  this  possibility. 

The  review  quickly  noted  that  the  Defense  Advanced  Research  Projects  Agency  had  flown 
an  airframe,  not  a  complete  system.  Once  reliability  improvements  were  made  and  sensors, 
cameras,  and  communications  were  included,  the  cost  would  easily  become  $75  million  each. 

In  looking  for  ways  to  reduce  costs,  the  program  review  team  assumed  that  not  every 
Global  Hawk  had  to  do  every  mission.  Vehicle  configurations  could  be  tailored  to  different 


6  Interview  with  Defense  Contract  Management  Agency  staff  member,  2009  (name  withheld  by  request). 

7  Interview  with  retired  Air  Force  engineer,  2009  (name  withheld  by  request). 
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missions.  The  team  looked  at  requirements  and  discussed  with  the  user  how  to  relax  require¬ 
ments.  The  team  worked  with  the  Air  Force,  the  prime  contractor,  and  suppliers  to  explore 
what  could  be  done  to  improve  producibility,  including  adding  funding  to  reduce  recurring 
costs  and  generating  competition  for  avionics  suites. 

The  team  was  able  to  achieve  savings  from  proposed  prices  in  avionics. 

The  Air  Force  conducted  two  prominent  should-cost  reviews  in  2010.  Another  review  of 
the  Global  Hawk  program  was  led  by  the  Air  Force  Cost  Analysis  Agency.  The  other  promi¬ 
nent  review  was  directed  by  the  Secretary  of  the  Air  Force,  who  chose  the  Director  of  the  Air 
Force’s  Office  of  Small  Business  Programs  to  lead  a  team  to  conduct  a  review  of  the  Evolved 
Expendable  Launch  Vehicle  (EELV)  program.  Although  the  analytical  work  of  both  reviews 
has  been  completed  as  of  the  writing  of  this  report,  it  is  too  soon  to  determine  the  results  in 
terms  of  cost  reductions. 

The  2010  Global  Hawk  review  was  especially  unusual  in  that  the  team  was  small  and 
composed  mostly  of  cost  analysts.  They  identified  potential  savings  in  the  air  vehicle  and  pay- 
loads.  Subject-matter  experts  familiar  with  the  program  were  difficult  to  find. 

The  EELV  review,  by  contrast,  was  much  larger  and  more  diverse  in  the  composition  of  its 
team.  A  roster  of  participants  shows  100  members  assigned  to  the  direct  portion  of  the  review 
and  more  than  40  assigned  to  the  overhead  portion  of  the  review.  Six  more  members  were  from 
the  Defense  Acquisition  University  (DAU).  Based  on  interviews,  the  number  of  active  partici¬ 
pants  was  probably  smaller  than  the  total  number  of  people  on  the  roster.  Areas  of  expertise 
on  the  team  included  cost  analysis  and  estimating,  pricing,  contracting,  and  subject-matter 
expertise  in  avionics,  propulsion,  and  other  areas  related  to  launch  services  and  capabilities. 
The  team  members  represented  many  organizations  from  throughout  the  Air  Force  as  well  as 
NASA,  various  support  contractors,  the  Navy  Price  Fighters,  DCMA,  and  DAU. 

A  brief  orientation  on  should-cost  reviews  was  provided  at  the  beginning  of  the  effort  by 
DAU,  and  some  training  was  provided  by  one  of  the  support  contractors. 

The  bulk  of  the  effort  took  five  months  to  complete. 

One  of  the  participants  in  the  EELV  study  had  also  participated  in  similar  reviews  during 
the  1990s  of  other  large  weapon  system  programs.  He  said  that  although  the  reviews  were  not 
termed  should-cost  reviews,  they  were  done  fairly  routinely  for  large  weapon  system  programs 
in  the  Navy.  He  described  the  primary  methodology  used  on  the  EELV  review  as  auditing  or 
analysis  of  proposals  and  other  financial  data.  When  asked  about  other  methodologies  cited  in 
the  literature  or  by  other  advocates,  such  as  industrial  engineering,  he  claimed  that  although 
the  government  may  have  industrial  engineers  capable  of  the  analysis,  it  would  be  prohibi¬ 
tively  time-consuming  and  labor-intensive  for  government  engineers  to  become  familiar  with 
a  contractor’s  manufacturing  process  and  independently  evaluate  the  analysis.  He  said  that  on 
other  reviews,  including  those  of  the  C-17  and  V-22  programs,  the  prime  contractor  conducted 
industrial  engineering  studies  and  then  collaborated  with  the  government  should-cost  teams. 
In  addition,  the  C-17  and  V-22  contractors  were  conducting  ongoing  industrial  engineering 
reviews  and  manufacturing  improvement  efforts  to  reduce  costs.  These  were  distinct  efforts 
from  the  should-cost  reviews  but  were  used  by  the  should-cost  teams  in  the  development  of 
their  estimates.8 


Interview  with  Air  Force  Cost  Analysis  Agency  cost  estimator,  2010  (name  withheld  by  request). 
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Recent  Should-Cost  Experience  of  Other  DoD  Organizations  with  Air  Force  Programs 

We  also  spoke  with  people  in  two  organizations  outside  the  Air  Force  that  had  conducted 
should-cost  reviews  on  Air  Force  programs.  DoD’s  foremost  dedicated  capability  to  conduct 
program  should-cost  reviews  resides  with  the  Navy’s  Price  Fighters  organization.9  The  Price 
Fighters  workforce  consists  of  roughly  60  civilians,  mostly  industrial  technicians  and  engi¬ 
neers  with  cost  training,  assisted  by  support  contractors.  The  Price  Fighters  organization  is  an 
industrially  funded  activity  and  is  able  to  accept  funding  from,  and  work  for,  other  government 
organizations. 

The  Air  Force  has  hired  the  Navy  Price  Fighters  to  help  provide  negotiation  objectives 
in  several  cases.  Most  notably,  the  Price  Fighters  have  worked  on  the  C-130  center  wing  box, 
the  C-17  global  sustainment  program,  the  C-27  program,  and  the  EELV  should-cost,  as  men¬ 
tioned  above. 

Representatives  of  the  Price  Fighters  claimed  to  have  achieved  percentage  savings  from 
proposed  prices  in  the  high  teens  and  up.  They  explained  that  outcomes  vary  depending  on 
the  buying  organization.  Different  organizations  and  individuals  within  DoD  have  differing 
capabilities  and  appetites  to  negotiate  lower  prices,  but  the  Price  Fighters  can  enhance  these 
capabilities  with  actionable  information.10  This  insight  from  DoD’s  preeminent  organization 
for  conducting  should-cost  reviews  indicates  that  the  analytic  capability  is  only  part  of  the 
battle.  Achieving  real  savings  depends  on  additional  circumstances,  including  the  ability  and 
willingness  of  the  procuring  organization  to  negotiate  lower  prices.* 11 

A  much  more  limited  capability  for  should-cost  analysis  of  individual  spare  parts  exists  at 
the  Defense  Logistics  Agency’s  (DLA’s)  defense  supply  centers.  Small  groups  of  engineers  use 
various  techniques  to  analyze  proposed  prices  for  parts.  They  may  check  the  prices  of  previ¬ 
ous  buys  or  prices  for  similar  parts  on  other  platforms.  If  sufficient  technical  information  on 
the  part  is  available,  they  may  use  computer  software  models  to  estimate  costs  and  determine 
whether  individual  spare  parts  on  sustainment  programs  can  be  procured  more  cheaply. 

At  the  Aviation  Supply  Center  in  Richmond,  engineers  work  on  spare  parts  on  aircraft 
sustainment  programs.  Should-cost  estimates  are  tracked  to  determine  whether  the  contract¬ 
ing  officer  used  the  should-cost  estimate  in  the  buy;  if  so,  savings  from  proposed  prices  are 
calculated.  The  organization  in  Richmond  did  not  express  the  savings  in  percentage  terms,  but 
reported  that  it  has  achieved  a  high  return  on  investment  for  its  small  staff.12 


9  The  Price  Fighters  Department  is  located  within  Naval  Supply  Systems  Command,  Naval  Inventory  Control  Point. 
More  information  on  the  organization’s  staffing  and  training  is  provided  in  Chapter  Five. 

10  Interview  with  Navy  Price  Fighters  staff  members,  2010  (names  withheld  by  request). 

11  Various  participants  in  contract  negotiations  have  offered  several  potential  explanations  for  this.  Explanations  include 
inadequate  training  or  skill  in  negotiating,  aversion  to  conflict,  lack  of  interest  in  saving  money  if  enough  funds  are  bud¬ 
geted  to  execute  the  program,  and  a  desire  to  stay  on  good  terms  with  the  contractor  by  those  who  are  retiring  soon  from 
government  service  and  wish  to  work  for  the  contractor. 

12  Interview  with  Defense  Logistics  Agency,  Aviation  Supply  Center,  staff  member,  2009  (name  withheld  by  request). 
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Summary  of  Experiences 

Results 

We  found  scant  literature  addressing  the  question  of  whether  should-cost  reviews  result  in  sav¬ 
ings  compared  with  other  forms  of  contract  pricing  and  negotiation.  Such  comparisons  would 
provide  the  most  compelling  evidence  of  the  efficacy  of  should-cost  reviews.  Of  the  three  stud¬ 
ies  we  found  that  addressed  the  question,  two  found  no  significant  difference  in  results,  but 
the  studies  were  limited  by  very  small  sample  sizes.  The  1973  GAO  study  was  also  limited  by  a 
very  small  sample  size.  It  found  savings  in  negotiated  prices,  which  were  specified  on  only  one 
contract  and  which  did  not  reflect  final  costs  incurred  on  the  fixed-price  incentive  contracts. 
Opinions  expressed  during  interviews  regarding  savings  were  mixed.  The  savings  claimed  from 
should-cost  reviews  by  the  Navy  Price  Fighters  and  by  DLA’s  Aviation  Supply  Center  for  spare 
parts  are  substantial. 

Team  Composition  and  Methodology 

Should-cost  review  team  size  and  composition  vary  greatly  depending  on  the  scope  of  the 
review  and  size  of  the  program.  The  trend  toward  outsourcing  adds  complexity  by  increas¬ 
ing  the  number  of  contractors  with  significant  work  share,  locations,  and  accounting  systems. 
Team  size  and  mix  differ  depending  on  scope,  program  size,  and  number  of  subcontractors; 
whether  the  review  is  a  program  review,  an  indirect  review,  or  both;  and  other  considerations. 
Typical  disciplines  include  cost  estimating,  pricing,  contracting,  and  subject-matter  expertise 
in  the  technology  embodied  in  the  program  being  reviewed. 

The  analytical  methodologies  used  in  should-cost  reviews  vary  and  have  included  indus¬ 
trial  engineering,  parametric  cost  estimating,  auditing,  and  application  of  alternative  economic 
or  business  strategies.  Literature  on  early  experience  with  the  reviews  indicates  the  participa¬ 
tion  of  industrial  engineers  much  more  often  than  in  recent  experience.  Similarly,  literature  on 
early  experience  emphasizes  the  importance  of  the  role  of  contracting  officers,  who  ultimately 
use  the  knowledge  gained  from  the  reviews  in  contract  negotiations,  whereas  interviews  with 
participants  on  more  recent  efforts  suggest  less  participation  of  contracting  personnel. 

Finding  people  with  the  necessary  skills  who  are  highly  motivated  and  capable  can  be 
difficult.  The  numbers  of  people  with  skills  traditionally  used  in  should-cost  reviews,  such  as 
pricing  and  industrial  engineering,  has  decreased,  and  many  of  the  required  capabilities  are 
in  short  supply.  Compounding  the  difficulty  is  that  participation  on  a  should-cost  team  takes 
people  away  from  their  regular  duties  and  usually  requires  travel  and  extraordinary  workloads. 
Teams  have  often  augmented  organic  personnel  with  civilian  and  military  retirees,  federally 
funded  research  and  development  center  staff,  and  support  contractors. 

Organizational  Support 

Organizational  support  for  should-cost  reviews  can  be  thought  of  as  spanning  a  range.  At  a 
high  level,  political  pressure  or  acquisition  strategies  may  align  powerful  forces  in  support  of 
a  contractor  or  weapon  system.  At  this  level,  organizational  support  may  take  the  form  of  the 
willingness  of  the  Air  Force  to  reduce  the  cost  of  a  program  or  overhead  cost  structure  at  a  con¬ 
tractor  facility  or  even  cancel  a  program.  At  a  low  level,  organizational  support  takes  the  form 
of  providing  the  resources,  in  the  form  of  people  and  training,  to  do  the  analysis. 

At  a  high  level  of  organizational  support,  should-cost  reviews  put  the  government  in  a 
difficult  position  of  having  to  evaluate  the  efficiency  and  reasonableness  of  private  contracts 
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and  labor  agreements.  A  program  should-cost  review  may  estimate  the  hours,  labor  mix,  and 
methods  required  for  production.  An  overhead  should-cost  review  may  examine  contractor 
employee  compensation,  benefits,  and  retirement  plans.  Assessing  compensation  is  always  sen¬ 
sitive.  There  is  a  tension  between  realism,  including  what  can  be  achieved  politically,  and  effi¬ 
ciency.  Savings  that  are  identified  in  principle  may  not  necessarily  be  achievable  in  practice. 
The  reviews  are  practical  exercises  that  are  influenced  by  the  exigencies  of  the  day.  For  all 
these  reasons,  clear  direction  and  top-level  support  by  the  Air  Force  leadership  are  essential  to 
achieving  results. 

Several  reviewers  of  an  earlier  draft  of  this  document  emphasized  that  success  was  more 
likely  when  the  government  had  the  leverage  of  a  credible  alternative,  such  as  the  ability  and 
willingness  to  cancel  the  program  or  at  least  reduce  proposed  buys.  This  is  an  example  of  a  high 
level  of  organizational  support. 

At  a  lower  level  of  organizational  support,  should-cost  teams  need  adequate  numbers  of 
people  with  the  right  skills  and  training.  It  is  not  clear  that  these  resources  are  being  provided 
for  recent  reviews.  The  composition  of  teams  has  changed  over  time,  with  less  participation 
by  industrial  engineers  and  pricers.  Team  compositions  may  have  changed  because  these  skills 
were  not  needed,  which  seems  unlikely,  or  because  personnel  in  these  small  and  specialized 
fields  were  not  available. 

Similarly,  recent  teams  have  received  little  training  on  should-cost  reviews.  The  EELY 
team  received  an  overview  of  the  concept  of  should-cost  reviews  from  DAU  and  some  training 
from  a  support  contractor.  The  Global  Hawk  and  F-35  teams  did  not  receive  training. 


CHAPTER  FOUR 


The  Use  of  Should-Cost  Analysis  and  Similar  Techniques  in 
Commercial  Businesses 


This  chapter  provides  a  brief  discussion  of  the  use  of  should-cost  analysis  and  similar  tech¬ 
niques  in  commercial  businesses.  This  review  of  commercial  practices  is  included  in  the  report 
for  two  reasons.  First,  most  of  the  literature  on  should-cost  analysis  within  DoD  is  old,  whereas 
there  is  a  rich  literature  on  commercial  practices  that  reflects  current  practices.  Second,  it  is 
plausible  that  the  current  ways  of  thinking  about  relationships  with  suppliers  and  the  use  of 
should-cost  and  similar  techniques  within  DoD  have  not  kept  pace  with  commercial  best  prac¬ 
tices,  and  that  notions  about  the  use  of  should-cost  analysis  within  DoD  could  benefit  from 
how  such  analyses  are  used  in  industry. 

This  review  of  the  literature  on  commercial  practices  is  intentionally  brief.  The  review  was 
limited  to  focus  narrowly  on  the  use  of  should-cost  analysis  and  similar  methods  in  commer¬ 
cial  businesses.  Not  all  commercial  practices  are  relevant  to  DoD,  thus  we  focus  here  on  those 
that  are  most  applicable. 


Differences  in  Buying  Environments  Between  DoD  and  Commercial  Business 

DoD  has  a  relationship  to  its  suppliers  that  is  unlike  the  relationship  of  most  commercial  orga¬ 
nizations  with  their  suppliers  in  a  few  important  ways,  and  it  is  important  to  keep  these  dif¬ 
ferences  in  mind  when  considering  lessons  from  commercial  experience.  One  key  difference  is 
that  most  commercial  organizations  compete  in  an  environment  in  which  they  face  constant 
market  pressure  to  improve  cost  and  performance.  These  pressures  provide  organizations  the 
motivation  to  seek  improvement  in  processes  and  products  from  themselves  and  their  sup¬ 
pliers.  In  contrast,  DoD  is  a  government-funded  entity  that  is  essentially  the  sole,  or  at  least 
dominant,  buyer  of  weapon  systems.1  DoD  generally  pays  a  single  prime  contractor  to  develop 
a  weapon  system  to  DoD  specifications,  then  buys  the  weapon  system  from  the  same  contrac¬ 
tor.  There  are  often  few  prime  contractors  capable  of  competing  for  the  development  of  large 
and  complex  weapon  systems,  and  once  the  development  contract  is  awarded,  there  is  seldom 
competition  for  the  production  contracts.  Pressure  to  control  cost  varies  depending  on  the 


1  Many  U.S.  weapons  systems  also  have  foreign  military  customers  that  buy  the  product  after  it  has  been  developed  for 
DoD,  and  generally  after  DoD  has  bought  the  earliest  and  most  expensive  production  units.  The  government  reviews  and 
approves  these  sales  to  ensure  they  are  in  accordance  with  U.S.  foreign  policy  objectives  and  to  protect  sensitive  technol¬ 
ogy.  In  rare  cases,  such  as  the  F-35  program,  foreign  militaries  contribute  to  the  development  of  a  weapon  system  that  is 
predominantly  funded  by  the  United  States.  Nevertheless,  DoD  has  by  far  the  largest  budget  of  any  buyer  of  U.S.-made 
weapon  systems. 
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political  environment,  and  those  pressures  may  or  may  not  affect  individual  program  offices 
or  procuring  organizations,  depending  on  such  circumstances  as  the  generosity  of  the  budget 
for  the  program  or  popularity  of  the  program,  so  that  participants  may  feel  less  motivation  to 
control  costs  than  in  the  commercial  sector. 

A  second  difference  between  purchasing  in  DoD  and  in  commercial  organizations  is  that 
most  of  the  organizations  cited  in  the  literature  on  commercial  practices  have  engineering  and 
manufacturing  experience  and  expertise  in  the  product  they  produce,  and  they  are  able  to  use 
this  expertise  to  conduct  should-cost  and  similar  analyses  of  the  parts  of  their  products  that 
they  buy.  In  contrast,  DoD  does  not  build  weapon  systems.  Few  DoD  personnel  have  direct 
experience  with  the  design  or  manufacture  of  products,  although  the  government  does  have 
industrial  expertise  in  its  labs,  depots,  and  other  repair  organizations.  DoD  is  able  to  work 
around  this  problem  by  hiring  people  with  the  needed  skills  to  serve  on  should-cost  studies. 

A  third  difference  is  that  the  commercial  market  offers  relatively  clear  feedback  or  metrics 
on  the  success  of  products  and  cost-reduction  efforts.  It  is  more  difficult  to  measure  the  success 
of  government  products  or  services. 

These  differences  between  DoD  and  commercial  buying  environments  also  explain  why 
DoD  buying  practices  may  lag  behind  the  state  of  the  art  in  the  commercial  world.  For  this 
reason,  a  review  of  commercial  practices  can  be  thought-provoking. 

While  acknowledging  that  DoD  and  commercial  companies  operate  in  environments 
and  have  relationships  with  their  suppliers  that  differ  in  important  ways,  we  highlight  exam¬ 
ples  from  the  literature  on  commercial  practices  that  provide  insight  into  three  areas  relevant 
to  the  use  of  should-cost  in  DoD:  the  nature  of  the  relationship  with  suppliers  and  the  pricing 
approach  appropriate  to  the  relationship;  organizing  and  staffing  for  effective  buying;  and  the 
purposes  served  by  the  use  of  should-cost  analysis  and  similar  techniques. 


Clarification  of  Terms  Found  in  Commercial  Literature 

Before  sampling  the  literature,  it  is  helpful  to  clarify  the  terms  should-cost  analysis,  reverse  cost 
analysis,  and  target  costing  or  target  pricing,  which  are  found  in  the  literature  on  commercial 
practices.  In  the  discussion  of  DoD  should-cost  analysis,  we  have  focused  on  the  in-depth 
cost  analyses  conducted  by  multidisciplinary  teams  at  contractor  locations,  although  the  quick 
analyses  of  individual  spare  parts  prices  at  DLA  defense  supply  centers  was  also  noted.  The 
commercial  literature  includes  references  to  should-cost  analysis  of  both  these  types.  Reverse 
cost  analysis  is  useful  when  suppliers  are  reluctant  to  share  cost  information.  In  this  narrower 
type  of  analysis,  publicly  available  information  or  the  purchaser’s  engineering  judgment  may  be 
used  as  sources  of  information  on  supplier  costs.  This  method  is  known  as  reverse  cost  analysis 
or  as  should-cost  analysis  (Monczka,  Trent,  and  Handfield,  2001). 

Target  costing  or  target  pricing  is  another  type  of  cost  analysis  that  is  similar  to  should- 
cost  analysis.  Monczka,  Trent,  and  Flandfield  describe  target  pricing  as  a  method  used  in 
initial  product  development.  The  price  willing  to  be  paid  by  the  customer  is  an  input  to  the 
design  process  and  serves  as  a  constraint  on  how  much  the  product  can  cost.  The  cardinal  rule 
of  target  pricing  is  that  the  target  cost  must  never  be  violated  (Monczka,  Trent,  and  Handfield, 
2001,  p.  443).  This  approach  is  akin  to  the  cost  as  an  independent  variable  (CAIV)  approach 
used  in  DoD  during  the  1990s. 
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The  Nature  of  the  Relationship  with  Suppliers  and  the  Appropriate  Method 
of  Cost  Analysis 

Both  commercial  organizations  and  DoD  have  evolved  in  their  thinking  and  practices  regard¬ 
ing  the  nature  of  their  relationships  with  suppliers.  The  nature  of  the  relationship  is  important 
because  it  should  inform  the  types  of  cost  analyses  used  by  purchasing  organizations.  The  lit¬ 
erature  on  commercial  practices  provides  ideas  for  determining  the  appropriate  relationship. 

Ellram  (1996)  provides  a  conceptual  framework  for  determining  the  appropriate  method 
of  cost  analysis  for  a  purchase.  The  framework  classifies  purchases  along  two  dimensions: 
(1)  whether  they  are  ongoing  or  one-time  and  (2)  whether  the  relationship  sought  with  the  sup¬ 
plier  is  arm’s-length  or  a  strategic  alliance.  This  results  in  a  matrix  that  categorizes  four  types  of 
purchases.  For  purchases  from  suppliers  that  the  buyer  wishes  to  hold  at  arm’s-length,  one-time 
purchases  or  repetitive  low-dollar  purchases  are  considered  “low-impact,”  while  continuous 
purchases  are  considered  “leverage  purchases.”  For  purchases  from  suppliers  with  whom  the 
buyer  wishes  to  have  a  strategic  alliance,  one-time  purchases  are  considered  “critical  projects,” 
and  continuous  purchases  of  important  items  are  considered  “strategic  purchases”  (Ellram, 
1996,  Figure  1). 

Ellram  goes  on  to  identify  cost  analysis  techniques  appropriate  to  each  type  of  purchase. 
For  low-impact  and  leverage  purchases  from  suppliers  with  whom  the  purchaser  has  a  more 
arm’s-length  relationship,  the  techniques  are  simpler  and  make  more  use  of  publicly  available 
price  information  rather  than  shared  data.  Cost  analysis  techniques  appropriate  for  critical 
projects  and  strategic  alliances  are  more  sophisticated  and  make  more  use  of  shared  data.  In 
particular,  should-cost  analysis  is  identified  as  appropriate  for  leverage  purchases,  while  more 
collaborative  techniques,  such  as  target  costing,  are  appropriate  for  strategic  purchases  (Ellram, 
1996,  Figure  2), 

As  any  DoD  cost  estimator  or  contract  price  analyst  knows,  the  ability  to  conduct  cost 
analysis  depends  on  the  quality  and  availability  of  the  data.  This  is  true  in  DoD  as  well  as 
commercial  organizations.  Cost  data  will  be  more  difficult  to  obtain  when  the  buyer  and  seller 
have  a  more  distant  relationship  (Monczka,  Trent,  and  Handheld,  2001,  p.  443).  In  DoD, 
statutes  and  regulations  require  contractors  to  provide  detailed  cost  information  under  certain 
conditions. 

In  commercial  practice,  competitive  pressure  has  forced  most  purchasers  to  develop  closer 
relationships  with  their  suppliers  in  an  effort  to  continuously  improve  processes  and  costs.  The 
closer  relationships  have  required  a  willingness  to  share  cost  information.  “The  last  frontier  of 
information  exchange  between  firms  is  the  detailed  sharing  of  supplier  cost  data”  (Monczka, 
Trent,  and  Handheld,  2001,  p.  442).  Most  North  American  supply  hrms  have  been  reluctant 
to  share  this  information.  Despite  these  fears,  target-  and  cost-based  pricing  are  increasingly 
being  seen  as  benehcial  (p.  443). 

It  is  interesting  to  note  that  while  there  has  been  a  reluctance  to  share  cost  data  and  col¬ 
laborate  in  improving  process  among  suppliers  in  the  United  States,  world-class  manufactur¬ 
ers  elsewhere  have  practiced  these  techniques  for  decades.  Over  20  years  ago,  Womack,  Jones, 
and  Roos  documented  the  use  of  target  pricing  and  other  collaborate  practices  between  buying 
organizations  and  suppliers  in  the  Japanese  auto  industry  in  their  book  The  Machine  That 
Changed  the  World  (1991,  Chapter  6). 

An  example  of  collaborative  practices  between  a  buying  organization  and  its  suppliers  is 
Honda  of  America.  At  Honda,  suppliers  participate  in  the  development  of  new  Honda  prod- 
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ucts.  Engineers  from  suppliers  work  together  with  Honda  engineers  in  the  early  phase  of  vehi¬ 
cle  development  to  use  the  suppliers’  expertise  and  help  develop  the  best  design  in  the  shortest 
amount  of  time.  The  collaboration  includes  joint  efforts  in  cost  reduction  using  value  engineer¬ 
ing  and  other  techniques  (Honda  Supply  Team,  2011). 

It  is  also  interesting  to  note  that  long  after  some  world-class  manufacturers  adopted  closer 
collaboration  and  cost  sharing  among  suppliers,  and  while  such  practices  were  increasingly 
being  viewed  favorably  by  commercial  organizations  in  the  United  States,  legislation  and  prac¬ 
tice  within  DoD  during  the  1990s  and  early  2000s  were  moving  away  from  the  sharing  of 
cost  information.  Lorell,  Graser,  and  Cook  (2005)  describe  in  Price-Based  Acquisition:  Issues 
and  Challenges  for  Defense  Department  Procurement  of  Weapon  Systems  the  belief  popular  in  the 
1990s  that  use  of  market  pricing  rather  than  more  detailed  cost  information  would  reduce  the 
cost  and  acquisition  time  for  major  weapon  systems,  and  the  acquisition  reform  legislation  and 
DoD  practices  that  flowed  from  this  belief.  Several  years  after  the  implementation  of  these 
reforms,  the  authors  were  unable  to  find  evidence  of  meaningful  cost  savings  attributable  to  the 
implementation  of  price-based  acquisition  (Lorell,  Graser,  and  Cook,  2005,  p.  60). 


Organizing  for  More  Effective  Cost  Analysis  and  Buying  in  Supply  Chain 
Management 

It  is  also  instructive  to  note  how  commercial  entities  organize  themselves  to  become  more 
effective  buyers.  This  lesson  includes  how  effectiveness  is  measured  and  what  kind  of  people  or 
skills  are  used. 

Ellram’s  1999  study  examined  the  use  of  target  costing  at  1 1  organizations,  which  included 
manufacturing  and  nonmanufacturing  firms.  She  found  that  target  costing  is  best  performed 
by  cross-disciplinary  teams  (Ellram,  1999,  p.  12).  Primary  responsibility  for  the  target  costing 
process  within  the  organization  is  often  shared  between  purchasing  and  research  and  develop¬ 
ment  or  engineering  (p.  38). 

Ellram’s  2002  study  examined  best  practices  in  strategic  cost  management  at  five  large 
manufacturing  companies.  She  found  that  all  five  companies  had  a  centralized  or  mix  of  cen¬ 
tralized  and  decentralized  purchasing  organizations.  Cost  management  specialists,  either  from 
the  purchasing  and  supply  management  organization  or  from  finance,  were  the  focal  point  for 
analysis.  They  built  cost  models,  conducted  supplier  and  should-cost  analysis,  and  validated 
results  (Ellram,  2002,  p.  13).  The  organizations  made  sure  that  they  had  people  with  the  right 
expertise  involved,  and  this  generally  meant  finance  or  cost  management  people  (p.  14).  The 
techniques  used  include  benchmarking  of  prices  and  processes,  target  costing,  should-cost 
analysis,  and  total  cost  of  ownership.  Among  the  key  success  factors  to  strategic  cost  manage¬ 
ment  were  (pp.  17-18) 

•  high-level  visibility  and  reporting  relationships 

•  the  availability  of  trained  and  dedicated  personnel  to  support  supplier  cost  analysis 

•  credibility  of  the  numbers  reported 

•  cost  management  as  a  priority  throughout  the  organization 

•  collection  of  cost  data,  and  information  systems  with  the  data  to  allow  analysis. 
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Fearon  and  Bales  (1997)  studied  what  measures  of  purchasing  effectiveness  were  consid¬ 
ered  to  be  useful  by  chief  procurement  officers  (CPOs)  and  chief  executive  officers  (CEOs), 
as  well  as  which  measures  were  actually  used  by  CPOs.  CEOs  and  CPOs  provided  separate 
responses  for  the  measures  that  were  assessed  as  being  useful  and  which  ones  were  actually 
used.  When  assessed  value  and  usage  were  combined,  the  three  highest-rated  measures  were 
(1)  price  negotiations  resulting  in  savings,  (2)  use  of  leverage  through  combining  volumes,  and 
(3)  past  delivery  performance  (p.  11). 

In  the  Supply  Performance  Management  Benchmarking  Report  (2005)  issued  by  CAPS 
Research,  38  companies  rated  the  effectiveness  of  strategies  that  helped  achieve  cost  reductions. 
Should-cost  analysis  received  a  mean  score  of  2.57  on  a  scale  from  1  (not  effective)  to  5  (very 
effective).  Other  strategies,  such  as  re-sourcing/competitive  bidding  and  re-negotiations,  were 
deemed  more  effective.  On  average,  over  half  of  supplier  products  were  multiple-sourced  by  the 
companies  that  responded,  with  an  additional  roughly  one-third  using  single  sourcing.  Only 
roughly  11  percent  of  the  supplier  products  were  sole-sourced.2 


Examples  of  the  Use  and  Purposes  of  Should-Cost  and  Similar  Techniques 

The  following  examples  illustrate  the  use  of  should-cost  and  similar  techniques  in  diverse 
industries  and  the  purposes  for  which  the  analyses  are  conducted. 

The  1970  GAO  report  Feasibility  of  Using  “Should  Cost”  Concepts  in  Government  Procure¬ 
ment  and  Auditing  described  the  use  of  should-cost  concepts  in  industry  and  found  that  they 
were  used  pre-award  and  post-award  and  that  it  was  not  uncommon  to  send  engineers  and 
other  technical  experts  to  a  supplier’s  facility  to  help  make  operations  more  efficient.  Cost  and 
technical  data  reporting  requirements  imposed  by  commercial  organizations  on  their  suppliers 
were  extensive  in  many  cases  (GAO,  1970). 

More  recently,  Ellram  (2002)  conducted  detailed  case  studies  of  strategic  cost  manage¬ 
ment  in  five  large  manufacturing  firms.  Should-cost  analysis  was  one  of  the  cost  tools  used. 
She  found  that  should-cost  analysis  is  used  for  purchases  of  all  types  and  is  used  “to  facilitate 
improvement  both  within  the  organization  and  with  suppliers,  to  increase  the  organization’s 
understanding  of  costs,  as  a  tool  to  work  more  closely  with  suppliers,  and  to  help  support  evalu¬ 
ation  of  other  cost  analysis  approaches,  external  data,  bids,  and  other  items”  (p.  14). 

Other  research  describes  how  companies  use  should-costing  to  determine  what  they 
should  pay  their  suppliers  for  products.  Smock,  Rudzky,  and  Rogers  (2007)  describe  the  use  of 
should-costing  for  this  purpose  by  forward-thinking  companies  in  general. 

Caterpillar  uses  should-costing  to  estimate  what  purchased  parts  ought  to  cost  and  how 
design  changes  might  affect  the  cost  (Manufacturing  Business  Technology,  2006). 

Bailey  et  al.  (2011)  describe  their  development  of  a  should-cost  model  for  full-truckload 
freight  shipments,  used  by  American  Airlines  to  assess  bid  quotes  from  full-truckload  carriers. 

Should-cost  modeling  is  used  in  the  natural  gas  drilling  industry  to  better  understand 
prices  bid  for  supplier  services  and  for  use  in  negotiations  with  the  suppliers  (Newton,  Cody, 
and  Carr,  2010). 


2  Single-sourcing  is  buying  from  one  supplier  when  others  are  available.  Sole-sourcing  is  when  only  one  supplier  for  a 
particular  item  is  available. 
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Monczka,  Trent,  and  Handfield  (2001)  write  that  important  features  of  cost-based  pric¬ 
ing  are  the  financial  incentives  offered  to  the  supplier  for  performance  better  than  that  agreed 
to  as  part  of  the  purchasing  agreement  (p.  444)  and  the  mutual  sharing  of  the  cost  savings 
(p.  445).  Thus,  better  performance  and  mutual  financial  benefits  are  purposes  in  conducting 
these  analyses. 

Commercial  manufacturers  including  Toyota,  Whirlpool,  and  LG  Electronics  use  the 
similar  technique  of  target  costing  (Teague,  2009). 

Ellram  (2000)  sampled  11  companies  from  eight  industries  and  addressed,  among  other 
issues,  the  issue  of  why  organizations  use  target  costing.  She  found  that  the  reasons  include 
cost  reduction,  cost  disclosure  and  understanding,  continuous  improvement/competitiveness, 
improved  communications  and  early  involvement,  and  improved  design  and  accountability 
(p.  42). 


Key  Takeaways  from  Commercial  Experience  with  Should-Cost  and  Similar 
Analysis 

The  relationship  between  DoD  and  the  few  large  prime  contractors  from  which  it  buys  major 
weapon  systems  is  unusual,  and  possibly  even  unique.  When  we  try  to  learn  lessons  about 
buying  practices  from  commercial  organizations  and  apply  them  to  DoD,  we  must  keep  these 
differences  in  mind.  Despite  the  differences,  our  review  of  commercial  practices  offers  provoca¬ 
tive  insights  into  the  appropriate  nature  of  the  buying  relationship  between  DoD  and  its  sup¬ 
pliers,  the  purposes  for  which  the  analyses  can  be  used,  and  how  to  organize  and  staff  to  use 
should-cost  and  similar  analyses  effectively. 

The  Appropriate  Nature  of  the  Buying  Relationship  Between  DoD  and  Its  Suppliers 

A  central  tenet  of  DoD  acquisition  reform  in  the  1990s  was  the  belief  that  DoD’s  requirements 
for  detailed  cost  data  imposed  an  unusual  and  unnecessary  burden  on  defense  contractors  and 
that  removing  the  requirements  and  moving  to  a  price-based  acquisition  approach  would  save 
time  and  money.  A  review  of  DoD  acquisition  outcomes  found  no  evidence  of  meaningful 
savings  (Lorell,  Graser,  and  Cook,  2005).  The  review  of  commercial  practices  in  this  chapter 
reveals  that  DoD’s  requirements  for  detailed  cost  data  are  not  so  unusual.  Commercial  orga¬ 
nizations  use  cost  analysis  methods  with  their  suppliers  appropriate  to  the  nature  of  their  rela¬ 
tionship  with  the  supplier.  Detailed  information  on  cost  and  process  is  shared  in  commercial 
industry  when  relationships  with  suppliers  are  strategic  alliances.  For  weapon  systems  that  are 
made  by  contractors  specifically  for  DoD  and  to  its  specifications,  it  seems  appropriate  to  con¬ 
sider  these  relationships  strategic  alliances. 

The  Purposes  for  Which  Should-Cost  and  Similar  Analyses  Can  Be  Used 

The  sharing  of  detailed  data  and  expertise  between  supplier  and  purchaser  during  should-cost 
and  similar  analyses  serves  at  least  three  important  purposes  in  commercial  organizations. 
Similar  benefits  could  accrue  in  DoD. 

First,  the  purpose  of  should-cost  and  similar  analyses  is  to  allow  buying  organizations  to 
determine  how  much  they  should  pay  for  products  when  negotiating  with  suppliers. 

Second,  the  sharing  of  detailed  data  and  expertise  between  supplier  and  purchaser  during 
should-cost  and  similar  analyses  helps  improve  processes  and  practices,  which  results  in  cost 
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savings  for  both  parties.  The  participants  benefit  by  becoming  more  competitive  in  the  mar¬ 
ketplace,  and  ideally  more  profitable. 

A  third  important  purpose  of  should-cost  and  similar  analyses  is  to  gain  a  better  under¬ 
standing  of  the  costs  to  develop  and  produce  products  and  of  the  relationship  between  product 
attributes  and  costs.  This  understanding  is  especially  useful  during  the  development  of  new 
products,  when  commercial  firms  using  the  target  costing  approach  seek  to  develop  a  product 
that  will  be  affordable  to  the  customer. 

The  same  intention  of  target  costing  in  commercial  organizations  was  sought  by  DoD 
through  CAIV  and  remains  the  intention  of  DoD  in  the  “affordability  as  a  requirement”  policy 
for  product  development  (Carter,  2011).  Successful  implementation  of  this  intention  requires 
credible  shared  costs,  willingness  to  trade  between  cost  and  capability,  and  the  commitment 
to  not  violate  the  target  cost.  The  Joint  Strike  Fighter  (JSF)  program  is  probably  the  most  con¬ 
spicuous  current  example  of  the  difficulty  in  implementing  such  an  approach  in  DoD.  The  pro¬ 
gram  used  a  CAIV  approach  prior  to  system  development  and  demonstration  and  developed 
goals  for  the  procurement  cost  of  the  aircraft.  In  the  Selected  Acquisition  Reports  prepared  by 
DoD  and  sent  to  Congress,  DoD  described  the  program  through  2009: 

The  cornerstone  of  the  JSF  Program  is  affordability — reducing  the  development  cost,  pro¬ 
duction  cost,  and  cost  of  ownership  of  the  JSF  family  of  aircraft.  The  program  was  struc¬ 
tured  from  the  beginning  to  be  a  model  of  acquisition  reform,  with  an  emphasis  on  joint¬ 
ness,  technology  maturation  and  concept  demonstrations,  and  early  cost  and  performance 
trades  integral  to  the  weapon  system  requirements  definition  process.  (U.S.  Department  of 
Defense,  2009) 

However,  the  government  had  to  rely  heavily  on  industry  for  the  tools  that  informed 
trades  between  cost  and  capability  on  the  JSF,  leading  to  overly  optimistic  estimates  of  the 
capability  that  could  be  achieved  at  the  target  cost.  Once  the  program  entered  system  devel¬ 
opment  and  demonstration,  the  CAIV  approach  was  abandoned,  although  important  design 
decisions  continued  to  be  made  (Blickstein  et  ah,  2011).  Even  as  estimated  costs  for  the  product 
rose  to  over  50  percent  higher  than  the  original  estimates  and  threatened  the  program’s  afford¬ 
ability,  DoD  proved  reluctant  to  scale  back  its  requirements. 

Despite  DoD’s  difficulty  in  using  processes  such  as  CAIV  and  related  methods  to  acquire 
affordable  weapon  systems,  the  intention  and  similar  methods  such  as  target  costing  and 
should-cost  are  encompassed  by  OSD’s  will-cost/should-cost  initiative. 

The  final  section  of  the  chapter,  below,  summarizes  what  can  be  learned  from  commer¬ 
cial  experience  about  organizing  and  staffing  to  use  should-cost  and  related  methods  to  control 
costs. 

How  to  Organize  and  Staff  to  Use  Should-Cost  and  Similar  Analyses  Effectively 

The  studies  of  commercial  practices  reviewed  here  show  that,  similar  to  the  use  of  should- 
cost  in  the  Air  Force,  cross-disciplinary  review  teams  are  most  effective,  with  participation  of 
people  with  cost  or  finance  skills,  and  leadership  typically  from  the  purchasing  (contracting) 
organization  or  shared  with  another  organization.  Success  factors  are  similar  to  those  found 
in  Air  Force  use  of  should-cost  analysis  and  include  skilled  personnel,  management  support, 
and  adequate  cost  databases  and  information  systems.  The  effectiveness  of  should-cost  analy¬ 
sis  compared  with  other  strategies  appears  to  be  affected  by  the  degree  of  competition  among 
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suppliers,  with  other  strategies  such  as  competitive  bidding  or  re-sourcing  more  effective  when 
competition  permits. 

The  most  useful  and  highly  rated  measure  of  purchasing  effectiveness,  from  the  perspec¬ 
tive  of  corporate  leaders,  was  the  savings  resulting  from  negotiations. 

The  next  chapter  draws  on  the  experience  reviewed  in  previous  chapters  with  should-cost 
and  similar  reviews  in  the  Air  Force  and  in  commercial  organizations  to  explore  how  the  Air 
Force  could  enhance  its  capability  to  conduct  such  reviews. 


CHAPTER  FIVE 


How  Can  the  Air  Force  Enhance  Its  Capability  to  Conduct  Should- 
Cost  Reviews? 


This  chapter  on  should-cost  reviews  draws  lessons  learned  about  how,  why,  and  by  whom 
should-cost  reviews  were  conducted  for  the  purpose  of  enhancing  future  Air  Force  capability  to 
conduct  such  reviews.  The  discussion  assumes  that  DCMA  will  continue  to  have  lead  respon¬ 
sibility  for  the  overhead  portion  of  should-cost  reviews. 

The  lessons  drawn  in  this  section  are  derived  from  interviews  with  cost  analysts,  engi¬ 
neers,  and  price  analysts  who  participated  in  program  and  overhead  should-cost  reviews,  and 
from  literature  reviews. 


Existing  Should-Cost  Review  Capabilities 

As  a  baseline  from  which  to  consider  recommendations  for  enhancements  to  existing  capabili¬ 
ties,  it  is  helpful  to  understand  the  nature  of  those  capabilities.  The  Air  Force  has  no  dedicated 
organic  should-cost  capability,  although  it  does  have  individuals  with  the  required  skills  in 
key  areas,  including  cost  estimating,  pricing,  contracting,  and  industrial  engineering,  scat¬ 
tered  throughout  the  enterprise.  There  are  limited  numbers  of  people  in  some  or  even  most 
of  these  areas,  and  fewer  still  with  experience  with  should-cost.  Selected  other  organizations 
within  DoD  have  dedicated,  if  limited,  should-cost  capabilities.  The  Air  Force  can  make  use 
of  the  services  of  these  other  organizations,  and  these  capabilities  should  be  considered  when 
deciding  what  enhancements  the  Air  Force  should  make  to  enhance  its  existing  capabilities. 
For  example,  the  team  that  conducted  the  EELV  review  in  2010  was  composed  of  individuals 
with  diverse  skills  from  the  Air  Force,  the  Navy,  NASA,  the  National  Reconnaissance  Office, 
DCMA,  and  various  support  contractors.  This  does  not  prove,  however,  that  people  with  all 
the  desired  skills  and  experience  were  available,  or  that  the  Air  Force  would  be  able  to  conduct 
multiple  reviews  simultaneously  of  a  similar  scale. 

DoD’s  foremost  dedicated  capability  to  conduct  program  should-cost  reviews  resides  with 
the  Navy’s  Price  Fighters  organization.  The  Navy  Price  Fighters’  work  on  Air  Force  programs 
was  described  in  Chapter  Three.  It  is  helpful  to  understand  some  salient  features  of  that  orga¬ 
nization.  Representatives  from  the  Price  Fighters  mentioned  several  characteristics  of  their 
organization  as  strengths:1 

•  Most  of  the  people  have  hands-on  technical  expertise  with  the  technology  they  are 
analyzing. 


i 


Interview  with  Navy  Price  Fighters  staff  member,  2009  (name  withheld  by  request). 
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•  They  are  separate  from  any  buying  organization  and  thus  have  objectivity. 

•  They  are  dedicated  to  doing  should-cost  reviews  and  therefore  have  a  great  deal  of  experi¬ 
ence  and  expertise  with  them. 

•  They  are  familiar  with  the  handful  of  major  defense  prime  contractors  and  their  account¬ 
ing  systems  and  business  practices. 

•  They  understand  and  are  able  to  relate  to  program  management,  contract  and  pricing, 
and  technical  personnel. 

Management  of  the  organization  devotes  a  lot  of  attention  to  selecting  the  right  person¬ 
nel.  Roughly  half  of  the  people  in  the  organization  are  industrial  engineering  technicians, 
industry  specialists,  or  business  industry  specialists.  These  people  have  industrial  experience  in 
their  background — five  to  15  years  of  hands-on  experience  as  a  machinist  or  mechanic,  electri¬ 
cian,  tool  and  die  maker,  or  in  other  trades.  They  come  from  military  depots  or  shipyards,  as 
well  as  from  private  industry.  Another  roughly  one-quarter  of  the  staff  are  engineers.  All  staff 
receive  classroom  training  in  cost  analysis.  There  is  extensive  on-the-job  training  with  more 
experienced  staff.2 

Air  Force  customers  valued  the  Navy  Price  Fighters’  technical  or  subject-matter  expertise 
with  the  work  being  done  by  the  contractors,  cost  modeling  ability,  and  willingness  and  ability 
to  assist  contracting  officers  in  “going  toe-to-toe”  with  contractors  during  negotiations.3 

Within  DoD,  overhead  should-cost  review  capability  resides  primarily  within  DCMA. 
DCMA  forms  teams  to  conduct  reviews,  at  major  contractor  locations,  of  contractor  indirect 
costs,  elements  of  their  business  base,  rates,  and  other  areas  of  special  interest.4  Since  its  cre¬ 
ation  in  1990,  the  number  of  personnel  at  DCMA  (then  called  Defense  Contract  Management 
Command)  has  decreased  by  roughly  60  percent.  DCMA  is  currently  rebuilding  its  contract 
pricing  and  cost  monitoring  capabilities.  The  accounting  systems  of  major  defense  contractors 
are  extremely  complex,  so  DCMA  personnel  need  strong  accounting  and  auditing  skills  to  be 
effective  in  overhead  reviews.  The  capability  to  conduct  overhead  should-cost  reviews  was  not 
assessed  as  part  of  this  research. 


Key  Workforce  Capabilities  Required  for  Should-Cost  Reviews 

The  key  participants  on  a  program  should-cost  review  depend  to  some  extent  on  whether 
the  program  is  in  development  or  production.  Key  members  on  reviews  of  development  and 
production  programs  in  the  past  two  decades  are  cost  estimators,  engineers  with  expertise  in 
the  technical  disciplines  involved,  and  contracting  personnel  to  arrange  for  the  appropriate 
contracting  vehicle  to  achieve  cost  savings.  Manufacturing  personnel  are  especially  useful  in 
production  programs,  as  are  test  personnel  in  development  programs.  In  some  cases  it  may  be 
beneficial  to  include  a  program  manager  with  skills  in  business  strategies  to  incentivize  the 


2  Interview  with  Navy  Price  Fighters  staff  members,  2010  (names  withheld  by  request). 

3  Interview  with  Air  Logistics  Center,  Contracting  Directorate,  staff  member,  2009  (name  withheld  by  request). 

4  The  Air  Force  used  to  have  this  capability  in  its  Air  Force  Contract  Management  Division  until  the  organization  was 
disestablished  in  1990  and  its  contract  management  functions  were  transferred  to  the  newly  created  Defense  Contract  Man¬ 
agement  Command  within  DLA.  Defense  Contract  Management  Command  has  since  been  renamed  Defense  Contract 
Management  Agency  and  made  separate  from  DLA. 
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contractor.5  Similarly,  pricers  should  be  key  participants  if  the  reviews  are  to  be  used  as  the 
basis  for  contract  negotiations. 

Fitzgerald  maintains  that  industrial  engineers  are  needed  to  estimate  labor  based  on 
standard  work  hours.  The  Air  Force’s  now-obsolete  Should  Cost  pamphlet,  AFP  70-5,  advised 
the  same  approach.  However,  Fitzgerald  argues  that  the  entire  discipline  has  been  deempha- 
sized,  that  industrial  engineering  practices  are  declining  in  the  aerospace  industry,  and  that  the 
increasing  prices  for  aircraft  over  time  are  a  result.6 

The  Air  Force  has  organic  personnel  with  the  individual  skills  needed  to  participate  in 
should-cost  reviews,  but  questions  remain  regarding  whether  there  are  enough  people  in  key 
disciplines,  whether  they  have  knowledge  or  background  in  the  programs  being  reviewed,  and 
whether  they  are  available  to  work  on  reviews  that  interrupt  their  regular  duties  and  probably 
require  significant  travel.  The  numbers  of  Air  Force  personnel  in  some  of  the  disciplines  are 
small.  For  example,  there  were  only  164  pricers  in  the  Air  Force  at  the  end  of  FY  2009  and 
only  160  industrial  engineers,  with  135  of  the  latter  at  just  three  locations — Tinker,  Robins, 
and  Wright-Patterson  Air  Force  bases  (Air  Force  Personnel  Center,  2012). 

More  generically,  the  Air  Force  has  several  thousand  engineers  of  various  types,  but  the 
numbers  alone  do  not  indicate  how  many  have  subject-matter  expertise  in  programs  that  may 
be  candidates  for  should-cost  reviews.  Nor  do  the  numbers  indicate  experience  with  reviews  of 
this  nature  or  the  ability  or  willingness  to  participate  in  them. 

The  only  thing  that  is  clear  from  examining  Air  Force  and  commercial  experience  is  that 
a  cross-functional  team  of  various  skills  is  needed  to  conduct  should-cost  reviews.  Given  the 
mixed  record  of  success  with  achieving  savings,  and  the  changes  in  the  nature  of  acquisition 
programs  (particularly  the  trend  to  outsourcing),  it  is  not  clear  what  mix  of  personnel  would 
be  most  effective  in  current  acquisition  programs.  Recommendations  in  Chapter  Six  address 
this  question. 

Total  Number  of  People  Required  to  Conduct  Should-Cost  Reviews 

The  total  number  of  people  required  to  establish  a  dedicated  capability  for  should-cost  review  is 
determined  by  the  size  of  should-cost  teams  and  the  demand  for  large  should-cost  reviews.  The 
size  of  individual  teams  involved  in  the  reviews  discussed  in  this  report  varied,  from  roughly  a 
dozen  on  some  reviews  to  80  to  100  on  the  B-2  review  in  the  1980s  and  over  100  on  the  EELV 
review.  The  B-2  team  included  those  doing  overhead  reviews  at  three  major  contractor  loca¬ 
tions.  Program  should-cost  review  teams  on  more  recent  and  typical  reviews,  such  as  for  the 
Joint  Primary  Aircraft  Training  System  and  Global  Hawk,  have  been  much  smaller. 

The  increasingly  dispersed  nature  of  production  of  major  weapon  systems  also  affects  the 
size  of  individual  teams.  For  aircraft  systems,  the  engines  and  avionics  systems  have  for  a  long 
time  been  manufactured  by  suppliers  to  prime  contractors.  Airframe  fabrication  and  assembly, 
as  well  as  most  integration,  used  to  be  performed  largely  at  a  single  prime  contractor  site.  The 
production  process  for  many  airframe  components  was  more  consolidated;  prime  contractors 
would  fabricate  airframe  parts  in  one  location  on  a  factory  floor,  wire  and  assemble  in  an  adja¬ 
cent  location,  and  complete  final  assembly  nearby.  The  current  production  processes  are  gener- 


5  Interview  with  retired  Air  Force  engineer,  2009  (name  withheld  by  request). 

6  Interview  with  A.  Ernest  Fitzgerald,  retired  Air  Force  Financial  Management  and  Comptroller  staff  member,  2009. 
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ally  far  more  outsourced.  Fabricated  parts  from  a  third-  or  fourth-tier  supplier  may  be  sent  to  a 
second-tier  supplier  for  wiring  and  assembly.  These  parts  are  then  assembled  at  the  prime  site. 

The  diversification  of  industry  supply  chains  across  different  enterprises  and  locations 
likely  increases  the  expense,  time,  difficulty,  and  number  of  people  needed  to  conduct  thor¬ 
ough  should-cost  reviews.  The  direct  production  labor  at  the  prime  contractor  represents  a 
small  fraction  of  the  cost  of  most  programs,  which  implies  that  it  should  not  be  the  focus  of 
analytic  effort  during  the  review.  The  trend  to  outsourcing  changes  the  focus  of  the  review  to 
the  upstream  supply  chain  and  how  the  prime  contractor  manages  it  rather  than  the  prime 
contractor’s  in-house  manufacturing.  This  shift  may  require  different  skills  for  participants  as 
well.  Programs  including  the  C-17,  F-22,  and  F-35  embody  high  levels  of  outsourcing.  It  would 
be  useful  to  examine  the  experience  of  the  review  teams  on  these  programs  to  determine  the 
level  of  effort  and  skill  mix  required  due  to  outsourcing. 

Demand  for  reviews  will  depend  on  the  number  of  large  acquisition  programs.  By  the  FAR 
definition,  demand  would  depend  more  strictly  on  the  number  of  large  acquisition  programs 
entering  production.  The  guidance  from  USD(AT&T)  dramatically  expands  the  demand  for 
reviews,  mandating  that  they  be  done  for  ACAT  I,  II,  and  III  programs.  The  November  3, 
2010  guidance  directs  that  ‘“Should  cost’  targets  will  be  developed  using  sound  estimating 
techniques  that  are  based  on  bottom  up  assessments  of  what  programs  should  cost,  if  reason¬ 
able  efficiency  and  productivity  enhancing  efforts  are  undertaken.  These  costs  will  be  used  as 
a  basis  for  contract  negotiations  .  .  .”  (Carter,  2010b). 

Organization  for  a  Should-Cost  Review  Capability 

Various  organizational  structures  could  be  used  to  support  a  should-cost  review  capability 
within  the  Air  Force. 

The  first  approach  is  the  current  ad  hoc  arrangement,  in  which  there  is  no  dedicated  capa¬ 
bility  and  ad  hoc  teams  are  formed  at  the  direction  of  high-level  officials,  such  as  the  Assistant 
Secretary  of  the  Air  Force  for  Acquisition. 

A  second,  somewhat  more  structured  capability  would  be  a  skeletal  or  “virtual”  organiza¬ 
tion,  which  would  notionally  include  a  capability  to  enable  the  rapid  staffing  and  training  of 
ad  hoc  should-cost  teams  and  capture  lessons  learned. 

A  third,  more  robust  possible  organizational  structure  would  consist  of  a  small  cadre  of 
personnel  who  could  lead  should-cost  teams.  These  leaders  would  conduct  training,  lead  the 
should-cost  teams,  brief  results,  and  document  lessons  learned.  They  would  be  augmented  by 
specialists  in  required  fields  from  other  organizations. 

A  fourth  organizational  option  would  be  a  dedicated  capability  akin  to  the  Navy  Price 
Fighters  organization,  with  permanent  organic  staff  supplemented  by  support  contractor  staff 
and  federally  funded  research  and  development  centers  as  needed. 

Each  of  these  organizational  structures  has  merits  and  demerits.  The  current  arrange¬ 
ment,  with  no  dedicated  should-cost  capability,  is  the  least  expensive,  and  one  might  argue 
that  it  avoids  unnecessary  duplication  of  an  existing  dedicated  capability  within  the  Navy  that 
is  available  to  the  Air  Force. 

On  the  other  hand,  the  lack  of  a  dedicated  organization  has  serious  drawbacks.  Because 
should-cost  reviews  are  done  infrequently,  there  are  few  people  in  the  Air  Force  who  have 
done  them,  and  because  they  are  staffed  on  an  ad  hoc  basis,  the  few  experienced  people  are 
scattered  across  the  enterprise.  The  lack  of  a  dedicated  organization  results  in  a  loss  of  institu¬ 
tional  knowledge,  with  no  place  to  retain  records  or  people  with  experience.  This  arrangement 
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ensures  that  the  wheel  is  reinvented  with  each  review,  as  inexperienced  people  grapple  with 
how  to  perform  a  new  task.  Furthermore,  depending  on  the  level  of  the  tasking  and  involve¬ 
ment  by  the  official  directing  the  study,  it  can  be  difficult  to  find  and  compel  the  participation 
of  people  with  necessary  skills. 

A  skeletal  or  virtual  organization  could  alleviate  many  of  these  disadvantages.  Notionally, 
a  virtual  organization  would  consist  of  a  capability  to  enable  the  rapid  staffing  and  training  of 
ad  hoc  should-cost  teams.  This  would  include  a  contracting  capability  to  be  able  to  hire  retir¬ 
ees,  support  contractors,  federally  funded  research  and  development  centers,  or  another  gov¬ 
ernment  organization,  such  as  the  Navy  Price  Fighters,  if  organic  Air  Force  support  were  not 
available.  The  virtual  organization  would  exist  at  a  high  enough  level,  such  as  the  Office  of  the 
Assistant  Secretary  of  the  Air  Force  for  Acquisition,  to  have  enough  rank  to  direct  the  partici¬ 
pation  of  organizations  throughout  the  Air  Force.  The  organization  would  be  responsible  for 
developing  a  training  curriculum  on  should-cost  reviews  and  arranging  for  ad  hoc  teams  to  be 
trained  when  formed.  DAU  or  another  provider  could  develop  a  suitable  curriculum.  Results 
and  lessons  learned  from  each  review  would  be  documented. 

Compared  with  the  current  arrangement,  a  virtual  organization  would  incur  modest 
additional  costs.  It  would  facilitate  the  creation  of  teams  with  the  required  skills  by  broaden¬ 
ing  the  potential  pool  of  participants  and  providing  a  ready  mechanism  to  employ  them,  and 
it  would  enhance  their  ability  to  conduct  reviews  by  documenting  and  retaining  lessons  from 
previous  reviews  and  providing  training  to  new  teams. 

The  third  option  for  a  small  but  more  robust  capability  would  provide  the  advantages  of 
the  skeletal  organization  enumerated  above,  with  the  added  continuity  and  expertise  of  a  small 
permanent  cadre.  The  additional  advantage  would  come  at  the  added  expense  of  additional 
staff. 

These  three  options  have  the  advantage  of  adding  minimal  cost  and  personnel  authoriza¬ 
tions  compared  with  a  fully  staffed  and  dedicated  capability.  However,  the  less  robust  options 
have  potential  drawbacks.  Staffing  with  retirees  or  support  contractors  would  not  build  up  an 
organic  core  of  expertise,  and  the  skills  of  retirees  may  not  be  current.  In  addition,  to  the  extent 
that  there  are  not  enough  existing  organic  personnel  in  key  disciplines,  these  options  do  not 
increase  the  supply  of  scarce  organic  personnel. 

A  fourth  option  would  be  the  creation  of  a  dedicated  capability  to  conduct  should-cost 
reviews.  It  would  be  the  costliest  option.  The  specialized  skills  required  and  necessity  for  fre¬ 
quent  travel  imply  the  need  for  a  relatively  high  average  General  Schedule  (GS)  grade  level  to 
attract  the  appropriate  type  of  experienced  personnel  to  conduct  the  should-cost  reviews.  This 
option  would  be  appropriate  only  if  the  Air  Force  made  a  long-term  commitment  to  using 
should-cost  reviews  as  a  contract  negotiating  practice.  If  the  criteria  for  considering  a  review 
were  as  stated  in  the  FAR  (e.g.,  large,  sole-source  programs  with  some  initial  production  and  a 
history  of  increasing  costs),  ensuring  a  predictable  and  stable  workload  could  be  problematic. 
If  reviews  are  conducted  for  all  ACAT  I,  II,  and  III  programs  in  accordance  with  current  guid¬ 
ance,  the  issue  of  a  steady  workload  becomes  much  less  of  a  problem. 

A  dedicated  staff  would  have  important  advantages.  It  would  dedicate  otherwise  scarce 
skilled  personnel  to  conducting  should-cost  reviews,  so  that  availability  of  suitable  personnel 
would  not  be  a  problem  or  at  least  less  of  a  problem  than  it  is  currently.  A  dedicated  staff  would 
allow  the  development  of  institutional  knowledge  and  experience  in  this  specialized  skill. 

A  potential  disadvantage  is  that  a  dedicated  staff  performing  only  should-cost  reviews  may 
lose  touch  with  the  most  current  manufacturing  and  engineering  processes.  Another  potential 
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drawback  is  that  technical  staff  would  be  less  likely  to  be  subject-matter  experts  on  programs 
being  reviewed  than  technical  staff  drawn  from  a  program  office  or  related  organization.  Also, 
in  a  time  of  decreasing  budgets,  a  dedicated  staff  would  likely  require  taking  funding  and  bil¬ 
lets  from  other  functions. 

Another  important  consideration  regarding  a  should-cost  capability  is  its  location  within 
the  Air  Force.  Commercial  best  practices  indicate  that  such  functions  are  usually  led  by  cost/ 
financial  management  or  contracting  personnel  and  that  top-level  support  in  the  organization 
is  needed  for  success.  The  need  for  high-level  organizational  support  for  success  indicates  that 
a  location  in  either  the  office  of  the  Assistant  Secretary  of  the  Air  Force  for  Financial  Manage¬ 
ment  or  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition  would  be  appropriate. 

Table  5.1  summarizes  the  organizational  options  presented. 


Table  5.1 

Advantages  and  Disadvantages  of  Organizational  Options  for  an  Air  Force 
Should-Cost  Capability 


Organizational 

Option 

Advantages 

Disadvantages 

Ad  hoc 

Least  costly 

Least  capable 

Loss  of  institutional  knowledge 

Skeletal 
or  virtual 

Inexpensive 

Develops  training 

Captures  lessons  learned 

Allows  quick  hiring 

No  permanent  substantive  staff  to 
build  expertise 

Cadre 

Same  as  skeletal,  plus 

More  personnel  billets  and  cost 

Builds  core  leadership 

Staff  burnout 

Dedicated 

Same  as  cadre,  plus 

Costliest 

capability 

Staff  availability  and  experience 

Staff  could  lose  touch 

CHAPTER  SIX 


Recommendations  and  Conclusions 


This  study  examined  the  capability  to  conduct  should-cost  reviews  in  the  Air  Force.  Several 
key  findings  emerged: 

•  Should-cost  reviews  are  a  special  form  of  cost  analysis  described  by  the  FAR,  and  accord¬ 
ing  to  the  FAR  are  to  be  used  as  a  tool  in  contract  negotiations 

•  Although  should-cost  reviews  can  identify  inefficiencies  and  potential  ways  to  reduce 
costs,  this  does  not  automatically  translate  to  savings.  The  Air  Force  must  be  willing  and 
able  to  use  the  results  of  the  analyses  to  negotiate  lower  prices  on  contracts  and  must 
structure  and  administer  the  contracts  so  that  the  final  prices  do  not  increase  and  negate 
the  originally  negotiated  prices. 

•  The  FAR  specifies  that  program  should-cost  reviews  should  be  considered  in  certain  cir¬ 
cumstances,  especially  for  sole-source  production  of  major  acquisitions. 

•  There  is  a  lack  of  evidence  that  should-cost  reviews  save  money  compared  with  other 
forms  of  contract  pricing  and  negotiation,  as  well  as  a  lack  of  evidence  about  key  factors 
or  circumstances  that  determine  the  success  of  reviews. 

•  The  nature  of  production  processes  has  changed  since  the  heyday  of  should-cost  reviews 
in  DoD  in  the  1970s.  Today  most  of  the  manufacturing  of  weapon  systems  is  not  done 
by  prime  contractors  but  is  outsourced,  and  most  of  the  dollar  value  of  weapon  systems 
is  provided  by  suppliers.  Typically,  little  of  the  dollar  value  of  a  weapon  system  is  attrib¬ 
utable  to  direct  labor  at  the  prime  contractor.  These  changes  imply  a  different  focus  and 
different  techniques  for  should-cost  reviews  from  those  used  in  the  past. 

•  The  Air  Force  finds  people  to  conduct  should-cost  reviews  from  among  its  own  personnel, 
Air  Force  retirees,  other  government  agencies,  support  contractors,  and  federally  funded 
research  and  development  centers. 

•  The  Air  Force  does  not  have  a  dedicated  capability  to  conduct  should-cost  reviews,  which 
hampers  the  retention  of  institutionalized  knowledge,  experience,  and  lessons  learned. 

•  Few  Air  Force  personnel  have  experience  with  should-cost  reviews,  and  little  if  any  train¬ 
ing  in  how  to  do  them  has  been  provided  to  participants  in  recent  reviews.  Enhanced 
capability  to  do  should-cost  reviews  requires  training  for  all  participants. 


37 


38  The  Air  Force's  Experience  with  Should-Cost  Reviews  and  Options  for  Enhancing  Its  Capability  to  Conduct  Them 


Recommendations 

Four  recommendations  are  offered  to  enhance  the  Air  Force’s  capability  to  conduct  should-cost 
reviews: 

1.  Determine  whether  should-cost  reviews  in  the  Air  Force  result  in  savings  compared 
with  other  forms  of  contract  negotiation,  and  whether  there  are  factors,  such  as  meth¬ 
odology,  team  composition,  or  external  circumstances,  that  affect  success. 

2.  Assess  lessons  learned  from  recent  should-cost  reviews. 

3.  Develop  training  on  should-cost  reviews,  including  training  to  facilitate  interdisciplin¬ 
ary  exchange  on  multidisciplinary  teams. 

4.  Establish  databases  of  cost,  schedule,  earned  value,  and  technical  information  useful  for 
cost  estimating  and  pricing  activities. 

Determine  Whether  Should-Cost  Reviews  in  the  Air  Force  Result  in  Savings  Compared  with 
Other  Forms  of  Contract  Negotiation 

Price  negotiations  resulting  in  savings  were  the  single  most  valued  and  used  measure  of  pur¬ 
chasing  effectiveness  by  CPOs  and  CEOs  in  commercial  organizations  (Fearon  and  Bales, 
1997).  We  believe  it  is  a  useful  metric  for  the  Air  Force,  too,  and  therefore  our  first  recommen¬ 
dation  is  to  gather  and  analyze  the  information  to  compare  different  forms  of  cost  analysis  for 
contract  negotiation. 

The  RAND  research  team  found  little  evidence  that  indicates  that  any  of  the  various 
methods  of  contract  pricing  and  negotiation  in  the  Air  Force  achieved  better  results  than  the 
others.  Two  studies,  the  more  recent  from  1983,  did  formal  comparisons  of  matched  pairs  of 
Air  Force  contracts  (same  product  and  contractor,  one  contract  negotiated  with  should-cost 
and  one  without)  and  found  no  significant  difference  in  the  results.  Both  studies  suffered  from 
very  small  sample  sizes  that  limit  confidence  in  the  results.  A  GAO  study  from  1973  found 
greater  reductions  in  negotiated  prices  when  should-cost  was  used,  but  the  contract  prices  were 
not  final  at  time  of  the  study  and  therefore  did  not  necessarily  represent  savings.  Opinions 
expressed  in  interviews  about  the  savings  achieved  by  should-costs  were  not  based  on  formal 
structured  comparisons,  and  the  opinions  were  mixed. 

Some  participants  and  observers  consider  the  C-17  review  conducted  in  1994  to  be  most 
successful  of  those  in  recent  memory.  The  circumstances  surrounding  this  review  were  unusual 
in  that  the  Air  Force  had  identified  credible  alternatives  to  buying  additional  C-17s,  so  that  all 
participants  understood  that  the  future  of  the  program  was  at  stake.  Conversely,  interviewees 
describing  reviews  that  did  not  achieve  significant  savings  often  cited  circumstances  in  which 
the  Air  Force  had  no  good  alternatives.  These  interviews  at  least  suggest  a  hypothesis  that 
should  be  explored  in  a  more  formal  comparison  of  outcomes  of  different  pricing  and  negotia¬ 
tion  techniques. 

An  important  first  step,  then,  in  formulating  a  strategy  for  the  use  of  should-cost  reviews 
would  be  to  determine  whether  they  result  in  savings  compared  with  other  forms  of  contract 
negotiation,  under  what  circumstances,  and  using  what  methodologies.  It  might  be  the  case 
that  should-cost  reviews  are  successful  when  the  industrial  engineering  approach  is  used,  when 
a  contracting  officer  leads  the  team,  when  a  viable  alternative  exists  to  the  program  under 
review,  or  under  other  circumstances  or  using  other  processes.  Results  should  shape  workforce 
size,  skill  mix,  and  methodological  approaches.  Comparisons  of  already  completed  contracts 
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would  be  most  helpful  in  the  near  term.  Completed  contracts  would  capture  the  final  contract 
price  after  any  changes  that  may  have  occurred  during  execution. 

Assess  Lessons  Learned  from  Recent  Should-Cost  Reviews 

In  2010,  the  Air  Force  conducted  should-cost  reviews  of  the  EELV  and  Global  Hawk  pro¬ 
grams,  and,  in  2011,  DoD  conducted  a  should-cost  review  of  the  F-35  program,  reflecting 
a  resurgence  of  high-level  interest  within  DoD  in  using  should-cost  reviews  to  reduce  costs. 
Because  of  the  uneven  history  of  should-cost  reviews  on  large  programs,  the  lack  of  a  dedi¬ 
cated  capability  within  the  Air  Force  to  conduct  such  reviews,  and  the  shortage  of  people 
within  the  Air  Force  in  conducting  such  reviews,  we  recommend  that  an  assessment  of  lessons 
learned  from  these  and  any  other  recent  efforts  be  conducted.  The  assessment  should  address 
how  changes  in  the  manufacturing  of  modern  weapon  systems  affect  the  focus  and  techniques 
used  in  should-cost  reviews,  whether  people  with  the  required  skills  were  available  and  what 
additional  skills  would  be  needed  for  similar  reviews  in  the  future,  whether  sufficient  time  was 
available  for  the  reviews,  the  role  of  training  in  the  capability  to  conduct  the  reviews,  and  the 
outcome  of  the  reviews. 

Part  of  the  assessment  might  include  learning  lessons  from  organizations  recognized  as 
embodying  best  practices  in  this  area,  including  commercial  organizations,  and  benchmarking 
Air  Force  experience  against  their  methodologies,  organizational  structure,  and  staffing. 

On  the  EELV  and  Global  Hawk  reviews,  each  team  captured  lessons  learned.  This  is  a 
positive  step.  The  Air  Force  should  ensure  that  lessons  learned  from  reviews  are  captured  in  a 
consistent  fashion  and  documented,  retained,  and  made  available  for  training  future  teams. 

Develop  Training  on  Should-Cost  Reviews 

This  recommendation  is  contingent  on  the  outcome  of  the  previous  recommendation,  and 
on  the  content  of  the  should-cost  training  module  that  is  in  development  at  DAU  as  of  late 
FY  2011.  If  the  assessment  of  lessons  learned  finds  that  the  recently  conducted  should-cost 
reviews  had  positive  outcomes,  and  that  there  is  a  need  for  skill  development  to  be  able  to 
conduct  reviews  in  the  future,  the  Air  Force  should  invest  in  a  training  curriculum,  developed 
by  DAU  or  another  training  provider.  The  curriculum  should  be  shaped  by  findings  from  the 
lessons-learned  assessment  but  might  include  such  topics  as  how  to  select  should-cost  team 
members,  how  to  share  information  and  interact  on  a  multidisciplinary  team,  and  how  to 
translate  information  from  the  should-cost  estimate  into  a  lower  negotiated  contract  price. 
Because  should-cost  reviews  have  not  been  done  frequently  in  the  Air  Force  in  the  recent  past, 
most  participants  have  little,  if  any,  experience  with  the  process.  A  training  curriculum  could 
help  establish  a  repeatable  process  regardless  of  which  individuals  were  involved.  In  developing 
the  training,  the  training  provider  should  consider  training  practices  in  commercial  organiza¬ 
tions  that  use  should-cost  successfully. 

Contributions  from  multiple  disciplines  are  needed  to  accomplish  should-cost  reviews. 
In  particular,  a  solid  technical  assessment  of  a  program  and  the  labor  needed  to  perform  it 
is  the  foundation  of  this  work.  In  interviews  with  those  involved  in  the  acquisition  process, 
a  frequent  concern  was  the  quality  of  technical  inputs  from  subject-matter  experts.  Another 
frequent  concern  was  the  lack  of  understanding  of  the  motivations  or  of  the  contributions  of 
others  in  the  process.  For  example,  cost  estimators  and  pricers  often  need  the  same  data  but 
ask  for  them  in  different  ways,  in  different  formats,  or  over  different  time  periods  (such  as  cal¬ 
endar  year  versus  fiscal  year).  These  interdisciplinary  frictions  have  the  potential  to  degrade  the 
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quality  of  the  analyses.  For  example,  subject-matter  experts  may  not  be  able  to  convey  their 
knowledge  in  a  helpful  way  to  those  doing  cost/price  analysis,  or  cost/price  analysts  may  not  be 
able  to  convey  what  information  they  need  and  why. 

One  way  to  address  this  problem  is  with  training  between  members  of  different  func¬ 
tional  groups  who  work  together.  To  enhance  technical  assessments,  cost  estimators  and  pric¬ 
ers  could  be  trained  to  explain  to  subject-matter  experts  how  the  technical  input  will  be  used. 
Subject-matter  experts  could  be  trained  in  how  to  provide  assessments  that  are  useful  for  cost 
and  price  analysis.  Other  members  of  the  acquisition  workforce  might  be  trained  in  under¬ 
standing  the  needs  and  contributions  of  their  colleagues  from  different  functional  areas. 

Establish  Databases  of  Cost,  Schedule,  Earned  Value,  and  Technical  Information  Useful  for 
Cost  Estimating  and  Pricing  Activities 

An  industry  best  practice  recommended  for  adoption  by  the  Air  Force  is  the  creation  and 
maintenance  of  a  corporate-level  knowledge  database  that  can  be  used  for  contract  negotia¬ 
tions.  The  industry  best  practice  is  to  assign  people  the  responsibility  to  maintain  it  and  for  that 
duty  to  part  of  their  job  description  and  performance  appraisal. 

In  the  Air  Force,  relevant  data,  such  as  those  on  cost,  schedule,  earned  value,  and  techni¬ 
cal  issues,  are  collected  by  members  of  various  disciplines  or  organizations  but  are  often  not 
shared.  For  example,  schedule  and  earned  value  data  are  collected  by  individual  program  man¬ 
agers  but  not  routinely  shared  with  cost-estimating  organizations  that  could  use  the  data  to 
build  databases  and  develop  better  cost-estimating  relationships.  Analysts  attempting  to  assess 
how  much  labor  effort  will  be  required  to  perform  a  task  on  a  contract  are  unaware  of  similar 
tasks  and  labor  incurred  on  similar  previous  contracts.  This  stovepiping  of  information  handi¬ 
caps  Air  Force  acquisition  efforts  and  makes  each  acquisition  decision  less  informed  than  it 
could  have  been  if  acquisition  data  were  shared  across  the  enterprise. 

A  positive  example  of  establishing  and  sharing  a  cost-related  database  is  the  Air  Force 
Total  Ownership  Cost  database  (AFTOC).  AFTOC  contains  operating  and  support  costs  for 
Air  Force  systems  and  infrastructure.  As  an  additional  resource  to  cost  estimators,  AFTOC 
also  contains  aircraft  weight  statements  and  operational  test  reports  for  aircraft.  These  docu¬ 
ments  provide  technical  and  operational  characteristics  from  other  disciplines  that  are  often 
used  by  cost  estimators. 

Another  positive  example  of  an  enterprise-wide  database  is  the  Defense  Cost  and  Resource 
Center  (DCARC),  which  is  part  of  the  OSD  Office  of  Cost  Assessment  and  Program  Evalu¬ 
ation.  The  DCARC  contains  contractor  cost  data  reports  and  earned  value  reports  on  large 
acquisition  programs,  which  are  available  to  the  DoD  cost  analysis  community.  The  contractor 
cost  data  reports  on  large  programs  are  available  going  back  several  decades.  The  earned  value 
reports  are  available  only  over  the  past  few  years. 

While  the  cost  and  schedule  data  on  large  programs  collected  by  the  DCARC  are  an 
invaluable  resource,  the  Air  Force  would  benefit  from  a  similar  supplementary  database  on 
smaller  Air  Force  programs,  or  major  components  of  programs,  that  would  ideally  include  data 
on  aircraft  trainers  and  simulators,  support  equipment,  sustainment  contracts,  etc. 

The  exact  nature  of  the  database  and  the  data  should  be  informed  by  results  of  the  above 
recommendations.  The  Air  Force  should  first  determine  whether  its  should-cost  reviews  are 
effective,  using  what  methodologies  and  data,  and  under  what  conditions.  Commercial  prac¬ 
tices,  data  sources,  and  training  should  be  assessed.  Based  on  these  findings,  the  investment  in 
the  appropriate  database  can  be  made. 
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Conclusions 

Should-cost  analysis  as  described  in  the  FAR  is  a  specialized  form  of  cost  analysis,  used  to  sup¬ 
port  contract  negotiations,  that  is  characterized  by  a  focus  on  the  elimination  of  contractor 
inefficiencies.  It  is  significant  that  the  guidance  for  should-cost  analysis  is  found  in  the  federal 
regulation  for  the  contracting  function,  because  contracting  is  the  process  by  which  the  gov¬ 
ernment  specifies  what  it  wants  to  buy  and  at  what  price. 

It  is  a  difficult  task  to  identify  inefficiencies  and  potential  savings.  Doing  so  requires  the 
cooperation  of  the  prime  contractor  and  subcontractors  as  well  as  technical  expertise  that  is 
scarce  among  government  personnel.  But  the  potential  savings  will  only  be  achieved  if  the 
government  and  contractor  agree  to  eliminate  the  identified  inefficiencies  and  are  successful  in 
negotiating  a  contract  that  binds  the  contractor  to  a  lower  price.  The  negotiation  task  is  also 
difficult  and  requires  a  skilled  contracting  officer  who  can  use  the  results  of  the  should-cost 
review  to  inform  negotiations.  The  negotiation  task  is  made  more  difficult  because  the  govern¬ 
ment  is  frequently  in  the  position  of  wanting  a  product  that  is  available  from  only  one  supplier 
and  thus  is  negotiating  from  a  disadvantageous  position. 

The  tasks  of  identifying  potential  savings  and  negotiating  contracts  that  achieve  the 
potential  savings  require  technical  skills  that  can  be  learned  and  improved  by  Air  Force  acqui¬ 
sition  personnel,  if  given  the  right  tools.  But  it  will  take  more  than  technical  skills  to  achieve 
significant  savings  in  Air  Force  programs.  The  experience  assessed  here  suggests  that  should- 
cost  reviews  are  more  successful  when  supported  by  high-level  leadership  throughout  DoD  and 
when  the  government  has  credible  alternatives  to  the  program  in  question  that  strengthen  its 
bargaining  position.  These  additional  factors  for  success  may  present  a  greater  challenge  than 
conducting  a  should-cost  analysis. 
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